{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,24]],"date-time":"2026-06-24T11:22:27Z","timestamp":1782300147083,"version":"3.54.5"},"reference-count":31,"publisher":"MDPI AG","issue":"2","license":[{"start":{"date-parts":[[2024,1,22]],"date-time":"2024-01-22T00:00:00Z","timestamp":1705881600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100003052","name":"Ministry of Trade, Industry &amp; Energy (MOTIE, Korea)","doi-asserted-by":"publisher","award":["20014967"],"award-info":[{"award-number":["20014967"]}],"id":[{"id":"10.13039\/501100003052","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>The subtype diagnosis and severity classification of mood disorder have been made through the judgment of verified assistance tools and psychiatrists. Recently, however, many studies have been conducted using biomarker data collected from subjects to assist in diagnosis, and most studies use heart rate variability (HRV) data collected to understand the balance of the autonomic nervous system on statistical analysis methods to perform classification through statistical analysis. In this research, three mood disorder severity or subtype classification algorithms are presented through multimodal analysis of data on the collected heart-related data variables and hidden features from the variables of time and frequency domain of HRV. Comparing the classification performance of the statistical analysis widely used in existing major depressive disorder (MDD), anxiety disorder (AD), and bipolar disorder (BD) classification studies and the multimodality deep neural network analysis newly proposed in this study, it was confirmed that the severity or subtype classification accuracy performance of each disease improved by 0.118, 0.231, and 0.125 on average. Through the study, it was confirmed that deep learning analysis of biomarker data such as HRV can be applied as a primary identification and diagnosis aid for mental diseases, and that it can help to objectively diagnose psychiatrists in that it can confirm not only the diagnosed disease but also the current mood status.<\/jats:p>","DOI":"10.3390\/s24020715","type":"journal-article","created":{"date-parts":[[2024,1,22]],"date-time":"2024-01-22T12:01:13Z","timestamp":1705924873000},"page":"715","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":11,"title":["Mood Disorder Severity and Subtype Classification Using Multimodal Deep Neural Network Models"],"prefix":"10.3390","volume":"24","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-0261-6127","authenticated-orcid":false,"given":"Joo Hun","family":"Yoo","sequence":"first","affiliation":[{"name":"Department of Artificial Intelligence, Sungkyunkwan University, Suwon 16419, Republic of Korea"},{"name":"Hippo T&C Inc., Suwon 16419, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1613-0198","authenticated-orcid":false,"given":"Harim","family":"Jeong","sequence":"additional","affiliation":[{"name":"Hippo T&C Inc., Suwon 16419, Republic of Korea"},{"name":"Department of Interaction Science, Sungkyunkwan University, Seoul 03063, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ji Hyun","family":"An","sequence":"additional","affiliation":[{"name":"Department of Psychiatry, Sungkyunkwan University School of Medicine, Seoul 06351, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7687-8114","authenticated-orcid":false,"given":"Tai-Myoung","family":"Chung","sequence":"additional","affiliation":[{"name":"Hippo T&C Inc., Suwon 16419, Republic of Korea"},{"name":"Department of Computer Science and Engineering, Sungkyunkwan University, Suwon 16419, Republic of Korea"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2024,1,22]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"690130","DOI":"10.3389\/fpsyt.2021.690130","article-title":"Prevalence of suicidality in major depressive disorder: A systematic review and meta-analysis of comparative studies","volume":"12","author":"Cai","year":"2021","journal-title":"Front. Psychiatry"},{"key":"ref_2","unstructured":"Yonkers, K.A., and Samson, J.A. (2008). Handbook of Psychiatric Measures, American Psychiatric Publishing, Inc."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1111\/j.1440-1819.2010.02113.x","article-title":"Brain-derived neurotrophic factor as a biomarker for mood disorders: An historical overview and future directions","volume":"64","author":"Hashimoto","year":"2010","journal-title":"Psychiatry Clin. Neurosci."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"528","DOI":"10.1038\/mp.2013.25","article-title":"Clinical application of brain imaging for the diagnosis of mood disorders: The current state of play","volume":"18","author":"Savitz","year":"2013","journal-title":"Mol. Psychiatry"},{"key":"ref_5","unstructured":"Zhou, X.H., McClish, D.K., and Obuchowski, N.A. (2009). Statistical Methods in Diagnostic Medicine, John Wiley & Sons."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Pepe, M.S. (2003). The Statistical Evaluation of Medical Tests for Classification and Prediction, Oxford University Press.","DOI":"10.1093\/oso\/9780198509844.001.0001"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"71","DOI":"10.1016\/j.pnpbp.2017.06.024","article-title":"Application of machine learning classification for structural brain MRI in mood disorders: Critical review from a clinical perspective","volume":"80","author":"Kim","year":"2018","journal-title":"Prog.-Neuro-Psychopharmacol. Biol. Psychiatry"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1038\/s41398-021-01286-x","article-title":"Magnetic resonance imaging for individual prediction of treatment response in major depressive disorder: A systematic review and meta-analysis","volume":"11","author":"Cohen","year":"2021","journal-title":"Transl. Psychiatry"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"213","DOI":"10.1177\/1535370217750088","article-title":"Biomarker definitions and their applications","volume":"243","author":"Califf","year":"2018","journal-title":"Exp. Biol. Med."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1031","DOI":"10.1007\/s11517-006-0119-0","article-title":"Heart rate variability: A review","volume":"44","author":"Kannathal","year":"2006","journal-title":"Med. Biol. Eng. Comput."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"289","DOI":"10.1016\/j.ijcard.2006.11.221","article-title":"Heart rate variability in myocardial infarction and heart failure","volume":"120","author":"Chattipakorn","year":"2007","journal-title":"Int. J. Cardiol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"154","DOI":"10.1111\/j.1542-474X.2006.00097.x","article-title":"The relationship between heart rate recovery and heart rate variability in coronary artery disease","volume":"11","author":"Evrengul","year":"2006","journal-title":"Ann. Noninvasive Electrocardiol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"122","DOI":"10.1016\/j.ijcard.2009.09.543","article-title":"The relationship of autonomic imbalance, heart rate variability and cardiovascular disease risk factors","volume":"141","author":"Thayer","year":"2010","journal-title":"Int. J. Cardiol."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Pham, T., Lau, Z.J., Chen, S.A., and Makowski, D. (2021). Heart rate variability in psychology: A review of HRV indices and an analysis tutorial. Sensors, 21.","DOI":"10.20944\/preprints202105.0070.v1"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"S77","DOI":"10.1067\/mhj.2000.109981","article-title":"Heart rate variability in depressive and anxiety disorders","volume":"140","author":"Gorman","year":"2000","journal-title":"Am. Heart J."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"335","DOI":"10.1038\/s41398-019-0680-6","article-title":"The rise and fall of MRI studies in major depressive disorder","volume":"9","author":"Zhuo","year":"2019","journal-title":"Transl. Psychiatry"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"395","DOI":"10.1007\/s10844-021-00653-w","article-title":"Depression detection from sMRI and rs-fMRI images using machine learning","volume":"57","author":"Mousavian","year":"2021","journal-title":"J. Intell. Inf. Syst."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2211","DOI":"10.1109\/TNSRE.2021.3115266","article-title":"fNIRS evidence for distinguishing patients with major depression and healthy controls","volume":"29","author":"Chao","year":"2021","journal-title":"IEEE Trans. Neural Syst. Rehabil. Eng."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"735","DOI":"10.3389\/fpsyt.2018.00735","article-title":"Heart rate variability as indicator of clinical state in depression","volume":"9","author":"Hartmann","year":"2019","journal-title":"Front. Psychiatry"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Yoo, J.H., Son, H.M., Jeong, H., Jang, E.H., Kim, A.Y., Yu, H.Y., Jeon, H.J., and Chung, T.M. (2021, January 20\u201322). Personalized federated learning with clustering: Non-IID heart rate variability data application. Proceedings of the 2021 International Conference on Information and Communication Technology Convergence (ICTC), Jeju Island, Republic of Korea.","DOI":"10.1109\/ICTC52510.2021.9620852"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"e00633","DOI":"10.1002\/brb3.633","article-title":"Separating generalized anxiety disorder from major depression using clinical, hormonal, and structural MRI data: A multimodal machine learning study","volume":"7","author":"Hilbert","year":"2017","journal-title":"Brain Behav."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Shen, Z., Li, G., Fang, J., Zhong, H., Wang, J., Sun, Y., and Shen, X. (2022). Aberrated multidimensional EEG characteristics in patients with generalized anxiety disorder: A machine-learning based analysis framework. Sensors, 22.","DOI":"10.3390\/s22145420"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"73","DOI":"10.2528\/PIER20102202","article-title":"Distinguishing bipolar depression from major depressive disorder using fnirs and deep neural network","volume":"169","author":"Ma","year":"2020","journal-title":"Prog. Electromagn. Res."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1080\/14737175.2019.1550361","article-title":"Biomarkers for bipolar disorder: Current status and challenges ahead","volume":"19","author":"Teixeira","year":"2019","journal-title":"Expert Rev. Neurother."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1109\/TAFFC.2020.3047582","article-title":"A multi-modal stacked ensemble model for bipolar disorder classification","volume":"14","author":"AbaeiKoupaei","year":"2020","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Abaei, N., and Al Osman, H. (2020, January 4\u20138). A Hybrid Model for Bipolar Disorder Classification from Visual Information. Proceedings of the ICASSP, Barcelona, Spain.","DOI":"10.1109\/ICASSP40776.2020.9054648"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"117006","DOI":"10.1016\/j.eswa.2022.117006","article-title":"An overview of deep learning methods for multimodal medical data mining","volume":"200","author":"Behrad","year":"2022","journal-title":"Expert Syst. Appl."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3543848","article-title":"Multimodal classification: Current landscape, taxonomy and future directions","volume":"55","author":"Sleeman","year":"2022","journal-title":"ACM Comput. Surv."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"8279342","DOI":"10.1155\/2020\/8279342","article-title":"A review of multimodal medical image fusion techniques","volume":"2020","author":"Huang","year":"2020","journal-title":"Comput. Math. Methods Med."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Taleb, A., Lippert, C., Klein, T., and Nabi, M. (2021, January 28\u201330). Multimodal self-supervised learning for medical image analysis. Proceedings of the International Conference on Information Processing in Medical Imaging, Virtual.","DOI":"10.1007\/978-3-030-78191-0_51"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"162","DOI":"10.1109\/TRPMS.2018.2890359","article-title":"Deep learning-based image segmentation on multimodal medical imaging","volume":"3","author":"Guo","year":"2019","journal-title":"IEEE Trans. Radiat. Plasma Med. Sci."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/2\/715\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T13:47:30Z","timestamp":1760104050000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/2\/715"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,1,22]]},"references-count":31,"journal-issue":{"issue":"2","published-online":{"date-parts":[[2024,1]]}},"alternative-id":["s24020715"],"URL":"https:\/\/doi.org\/10.3390\/s24020715","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,1,22]]}}}