{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,28]],"date-time":"2026-08-28T00:18:24Z","timestamp":1787876304489,"version":"build-2784847793"},"reference-count":38,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2021,9,24]],"date-time":"2021-09-24T00:00:00Z","timestamp":1632441600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IJERPH"],"abstract":"<jats:p>Background: Online mental health information represents important resources for people living with mental health issues. Suitability of mental health information for effective self-care remains understudied, despite the increasing needs for more actionable mental health resources, especially among young people. Objective: We aimed to develop Bayesian machine learning classifiers as data-based decision aids for the assessment of the actionability of credible mental health information for people with mental health issues and diseases. Methods: We collected and classified creditable online health information on mental health issues into generic mental health (GEN) information and patient-specific (PAS) mental health information. GEN and PAS were both patient-oriented health resources developed by health authorities of mental health and public health promotion. GENs were non-classified online health information without indication of targeted readerships; PASs were developed purposefully for specific populations (young, elderly people, pregnant women, and men) as indicated by their website labels. To ensure the generalisability of our model, we chose to develop a sparse Bayesian machine learning classifier using Relevance Vector Machine (RVM). Results: Using optimisation and normalisation techniques, we developed a best-performing classifier through joint optimisation of natural language features and min-max normalisation of feature frequencies. The AUC (0.957), sensitivity (0.900), and specificity (0.953) of the best model were statistically higher (p &lt; 0.05) than other models using parallel optimisation of structural and semantic features with or without feature normalisation. We subsequently evaluated the diagnostic utility of our model in the clinic by comparing its positive (LR+) and negative likelihood ratios (LR\u2212) and 95% confidence intervals (95% C.I.) as we adjusted the probability thresholds with the range of 0.1 and 0.9. We found that the best pair of LR+ (18.031, 95% C.I.: 10.992, 29.577) and LR\u2212 (0.100, 95% C.I.: 0.068, 0.148) was found when the probability threshold was set to 0.45 associated with a sensitivity of 0.905 (95%: 0.867, 0.942) and specificity of 0.950 (95% C.I.: 0.925, 0.975). These statistical properties of our model suggested its applicability in the clinic. Conclusion: Our study found that PAS had significant advantage over GEN mental health information regarding information actionability, engagement, and suitability for specific populations with distinct mental health issues. GEN is more suitable for general mental health information acquisition, whereas PAS can effectively engage patients and provide more effective and needed self-care support. The Bayesian machine learning classifier developed provided automatic tools to support decision making in the clinic to identify more actionable resources, effective to support self-care among different populations.<\/jats:p>","DOI":"10.3390\/ijerph181910048","type":"journal-article","created":{"date-parts":[[2021,9,24]],"date-time":"2021-09-24T20:15:32Z","timestamp":1632514532000},"page":"10048","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Forecasting the Suitability of Online Mental Health Information for Effective Self-Care Developing Machine Learning Classifiers Using Natural Language Features"],"prefix":"10.3390","volume":"18","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7463-9208","authenticated-orcid":false,"given":"Meng","family":"Ji","sequence":"first","affiliation":[{"name":"School of Languages and Cultures, University of Sydney, Sydney 2006, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8528-5193","authenticated-orcid":false,"given":"Wenxiu","family":"Xie","sequence":"additional","affiliation":[{"name":"Department of Computer Science, City University of Hong Kong, Hong Kong 518057, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Riliu","family":"Huang","sequence":"additional","affiliation":[{"name":"School of Languages and Cultures, University of Sydney, Sydney 2006, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5819-4379","authenticated-orcid":false,"given":"Xiaobo","family":"Qian","sequence":"additional","affiliation":[{"name":"School of Computer Science, South China Normal University, Guangzhou 510631, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2021,9,24]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1016\/S2215-0366(18)30060-9","article-title":"Social determinants of mental disorders and the Sustainable Development Goals: A systematic review of reviews","volume":"5","author":"Lund","year":"2018","journal-title":"Lancet Psychiatry"},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"S27","DOI":"10.5694\/mja17.00972","article-title":"Identifying attributes of care that may improve cost-effectiveness in the youth mental health service system","volume":"207","author":"Hamilton","year":"2017","journal-title":"Med. J. Aust."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1619","DOI":"10.1017\/S0954579413000801","article-title":"Childhood exposure to violence and lifelong health: Clinical intervention science and stress-biology research join forces","volume":"25","author":"Moffitt","year":"2013","journal-title":"Dev. Psychopathol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"675","DOI":"10.1016\/j.puhe.2010.08.019","article-title":"Common mental disorders, unemployment and welfare benefits in England","volume":"124","author":"Ford","year":"2010","journal-title":"Public Health"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1007\/s00127-012-0515-8","article-title":"Feelings of loneliness among adults with mental disorder","volume":"48","author":"Meltzer","year":"2013","journal-title":"Soc. Psychiatry Psychiatr. Epidemiol."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"682","DOI":"10.1192\/bjp.2018.244","article-title":"Gender discrimination, victimisation and women\u2019s mental health","volume":"213","author":"Hosang","year":"2018","journal-title":"Br. J. Psychiatry"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"517","DOI":"10.1016\/j.socscimed.2010.04.027","article-title":"Poverty and common mental disorders in low and middle income countries: A systematic review","volume":"71","author":"Lund","year":"2010","journal-title":"Soc. Sci. Med."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"216","DOI":"10.1192\/bjp.bp.110.084764","article-title":"Factors associated with mental disorders in long-settled war refugees: Refugees from the former Yugoslavia in Germany, Italy and the UK","volume":"200","author":"Bogic","year":"2012","journal-title":"Br. J. Psychiatry"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"2540","DOI":"10.1001\/jama.298.21.2540","article-title":"Natural and Manmade Disasters and Mental Health","volume":"298","author":"Satcher","year":"2007","journal-title":"JAMA"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1037\/h0095162","article-title":"The development of the Mental Health Confidence Scale: A measure of self-efficacy in individuals diagnosed with mental disorders","volume":"23","author":"Carpinello","year":"2000","journal-title":"Psychiatr. Rehabil. J."},{"key":"ref_11","first-page":"20","article-title":"Family interventions for mental disorders: Efficacy and effectiveness","volume":"2","author":"Falloon","year":"2003","journal-title":"World Psychiatry"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"93","DOI":"10.1093\/oxfordjournals.schbul.a006862","article-title":"Family Support and Substance Use Outcomes for Persons with Mental Illness and Substance Use Disorders","volume":"27","author":"Clark","year":"2001","journal-title":"Schizophr. Bull."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"111","DOI":"10.1016\/j.jpainsymman.2011.03.013","article-title":"The Effectiveness of Patient-Family Carer (Couple) Inter-vention for the Management of Symptoms and Other Health-Related Problems in People Affected by Cancer: A Systematic Literature Search and Narrative Review","volume":"43","author":"Hopkinson","year":"2012","journal-title":"J. Pain Symptom Manag."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"346","DOI":"10.3109\/09638237.2011.648347","article-title":"Personalised computerized feedback in E-mental health","volume":"21","author":"Musiat","year":"2012","journal-title":"J. Ment. Health"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"263","DOI":"10.1192\/bjp.bp.113.142612","article-title":"Technological innovations in mental healthcare: Harnessing the digital revolution","volume":"206","author":"Hollis","year":"2015","journal-title":"Br. J. Psychiatry"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"857","DOI":"10.1016\/j.jad.2018.12.095","article-title":"Predicting persistent depressive symptoms in older adults: A machine learning approach to personalized mental healthcare","volume":"246","author":"Hatton","year":"2019","journal-title":"J. Affect. Disord."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1007\/s40737-020-00163-y","article-title":"From Learning About Machines to Machine Learning: Applications for Mental Health Rehabilitation","volume":"7","author":"Chaturvedi","year":"2020","journal-title":"J. Psy-chosoc. Rehabil. Ment. Health"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"603","DOI":"10.1002\/da.22646","article-title":"Sensing behavioral symptoms of mental health and delivering personalized in-terventions using mobile technologies","volume":"34","author":"Aung","year":"2017","journal-title":"Depress. Anxiety"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"649","DOI":"10.1017\/S1351324916000383","article-title":"Natural language processing in mental health applications using non-clinical texts","volume":"23","author":"Calvo","year":"2017","journal-title":"Nat. Lang. Eng."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"161","DOI":"10.14745\/ccdr.v46i06a02","article-title":"Challenges and opportunities for public health made possible by advances in natural language processing","volume":"46","author":"Baclic","year":"2020","journal-title":"Can. Commun. Dis. Rep."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"507","DOI":"10.1521\/suli.2005.35.5.507","article-title":"Writing Characteristics of Suicidal People on the Internet: A Psychological Investigation of Emerging Social Environments","volume":"35","author":"Barak","year":"2005","journal-title":"Suicide Life-Threatening Behav."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"279","DOI":"10.1002\/erv.521","article-title":"Use of text messaging in the aftercare of patients with bulimia nervosa","volume":"11","author":"Bauer","year":"2003","journal-title":"Eur. Eat. Disord. Rev."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"86","DOI":"10.1002\/mpr.1481","article-title":"Text mining applications in psychiatry: A systematic literature review: Text Mining Applications in Psychiatry","volume":"25","author":"Abbe","year":"2016","journal-title":"Int. J. Methods Psychiatr. Res."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"946","DOI":"10.1001\/jamadermatol.2016.1135","article-title":"Patient Education Materials in Dermatology: Addressing the Health Literacy Needs of Patients","volume":"152","author":"Prabhu","year":"2016","journal-title":"JAMA Dermatol."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"466","DOI":"10.1177\/0194599812442783","article-title":"Readability Assessment of Patient Education Materials from the American Academy of Otolaryngology\u2014Head and Neck Surgery Foundation","volume":"147","author":"Kasabwala","year":"2012","journal-title":"Otolaryngol.-Head Neck Surg."},{"key":"ref_26","unstructured":"Rayson, P., Archer, D., Piao, S., and McEnery, A.M. (2004, January 25\u201328). The UCREL semantic analysis system. Proceedings of the Beyond Named Entity Recognition Semantic Labeling for NLP Tasks Workshop, Lisbon, Portugal."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1515\/jlse.2010.009","article-title":"A corpus-based approach to mind style","volume":"39","author":"McIntyre","year":"2010","journal-title":"J. Lit. Semant."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"378","DOI":"10.1016\/j.csl.2004.11.002","article-title":"Comparing and combining a semantic tagger and a statistical tool for MWE extraction","volume":"19","author":"Piao","year":"2005","journal-title":"Comput. Speech Lang."},{"key":"ref_29","unstructured":"Tipping, M.E. (2000). The Relevance Vector Machine. Advances in Neural Information Processing Systems, Mit Press."},{"key":"ref_30","first-page":"211","article-title":"Sparse bayesian learning and the relevance vector machine","volume":"1","author":"Tipping","year":"2001","journal-title":"J. Mach. Learn. Res."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"1161","DOI":"10.1016\/j.ymssp.2009.10.011","article-title":"Application of relevance vector machine and logistic regression for machine deg-radation assessment","volume":"24","author":"Caesarendra","year":"2010","journal-title":"Mech. Syst. Signal Process."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"95","DOI":"10.1016\/j.asoc.2013.10.023","article-title":"Classification of silent speech using support vector machine and relevance vector machine","volume":"20","author":"Matsumoto","year":"2014","journal-title":"Appl. Soft Comput."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"99","DOI":"10.4103\/picr.PICR_41_18","article-title":"Understanding the properties of diagnostic tests\u2014Part 2: Likelihood ratios","volume":"9","author":"Ranganathan","year":"2018","journal-title":"Perspect. Clin. Res."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"168","DOI":"10.1136\/bmj.329.7458.168","article-title":"Diagnostic tests 4: Likelihood ratios","volume":"329","author":"Deeks","year":"2004","journal-title":"BMJ"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"1500","DOI":"10.1016\/S0140-6736(05)66422-7","article-title":"Refining clinical diagnosis with likelihood ratios","volume":"365","author":"Grimes","year":"2005","journal-title":"Lancet"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"763","DOI":"10.1016\/0895-4356(91)90128-V","article-title":"Likelihood ratios with confidence: Sample size estimation for diagnostic test studies","volume":"44","author":"Simel","year":"1991","journal-title":"J. Clin. Epidemiol."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"3122","DOI":"10.1111\/j.1572-0241.1999.01502.x","article-title":"The Usefulness of The Likelihood Ratio in The Diagnosis of Dyspepsia and Gastroesophageal Reflux Disease","volume":"94","author":"Moayyedi","year":"1999","journal-title":"Am. J. Gastroenterol."},{"key":"ref_38","unstructured":"Shreffler, J., and Huecker, M.R. (2020). Diagnostic Testing Accuracy: Sensitivity, Specificity, Predictive Values and Likelihood Ratios, StatPearls Publishing."}],"container-title":["International Journal of Environmental Research and Public Health"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1660-4601\/18\/19\/10048\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T07:04:28Z","timestamp":1760166268000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1660-4601\/18\/19\/10048"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,9,24]]},"references-count":38,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2021,10]]}},"alternative-id":["ijerph181910048"],"URL":"https:\/\/doi.org\/10.3390\/ijerph181910048","relation":{},"ISSN":["1660-4601"],"issn-type":[{"value":"1660-4601","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,9,24]]}}}