{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T22:10:00Z","timestamp":1790374200466,"version":"4.1.0"},"reference-count":42,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Ment Health"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Self-harm is the strongest risk factor for suicide and an important outcome for mental health care. Although prevalent in clinical populations, it is often imprecisely captured in routinely collected clinical data, where it is often recorded and stored as unstructured free text. Contemporary language models, such as GPT (OpenAI) and Gemini (Google), can analyze free-text clinical notes, but such models may violate data governance of processing sensitive patient data.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to evaluate whether a privacy-preserving language model running entirely within an institution\u2019s secure computing infrastructure (here, the UK National Health Service [NHS]) could accurately identify the presence and timing of self-harm using electronic health records from secondary mental health care.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n \n Clinical notes were drawn from Oxford Health NHS Foundation Trust using a multistage workflow: (1) a random sample of 1000 patients with a psychiatric diagnosis, defined according to the\n ICD-10<\/jats:italic>\n (\n International Statistical Classification of Diseases, Tenth Revision<\/jats:italic>\n ; codes F00\u2013F99); (2) candidate-note identification using a Gemma3-4b language model to flag notes containing self-harm content; and (3) from those candidates, 1352 randomly sampled notes were selected for expert annotation, resulting in gold-standard corpus enriched for self-harm content. Clinical notes were annotated for the presence of self-harm and its timing (\u226490 days, >90 days, or unknown). A privacy-preserving locally served 27-billion-parameter Gemma 3 language model (\u201cGemma3-27b\u201d) was used as the core model. Prompts were systematically developed and refined using a labeled development set to identify self-harm and generate a structured output per clinical record. Gemma3-27b performance was compared against a strong baseline multilabel text classification model based on robustly optimized BERT pretraining approach (RoBERTa), a transformer-based language model architecture. Model performance was evaluated using precision, recall, and the\n F<\/jats:italic>\n 1<\/jats:sub>\n -score (harmonic mean of precision and recall), with 95% CIs estimated from 1000 bootstrap samples with replacement.\n <\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n \n Gemma3-27b outperformed the RoBERTa classifier across all categories, achieving Precision=0.92, Recall=0.92 (sensitivity), and\n F<\/jats:italic>\n 1<\/jats:sub>\n -score=0.92 for notes containing self-harm, and Precision=0.97, Recall=0.97 (specificity), and\n F<\/jats:italic>\n 1<\/jats:sub>\n -score=0.97 for notes without self-harm. For the 51 notes labeled as recent self-harm in the held-out test set, Gemma3-27b achieved Precision=0.84, Recall=0.75, and\n F<\/jats:italic>\n 1<\/jats:sub>\n -score=0.79. The global weighted\n F<\/jats:italic>\n 1<\/jats:sub>\n -score of Gemma3-27b across all categories was 0.88, compared to 0.85 for RoBERTa.\n <\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n With systematic prompt development on a labeled development set, but no gradient-based fine-tuning, the current Gemma3-27b language model matched or exceeded a fine-tuned RoBERTa classifier for ascertaining self-harm events and their timing. Aggregate gains were modest, while improvements were largest in the most challenging, lower-frequency timing categories. On a simplified binary recent-versus-other task, RoBERTa performed marginally better, indicating that supervised classifiers remain highly effective when the task is simplified and sufficient labeled data exist. This work demonstrates the technical feasibility of privacy-preserving self-harm detection within a secure NHS research environment.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/87586","type":"journal-article","created":{"date-parts":[[2026,3,9]],"date-time":"2026-03-09T04:25:06Z","timestamp":1773030306000},"page":"e87586-e87586","source":"Crossref","is-referenced-by-count":1,"title":["Detection of Self-Harm in Electronic Mental Health Records Using Privacy-Preserving Local Language Models: Methodological Study"],"prefix":"10.2196","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3555-9181","authenticated-orcid":false,"given":"Andrey","family":"Kormilitzin","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9433-5340","authenticated-orcid":false,"given":"Dan W","family":"Joyce","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4662-8915","authenticated-orcid":false,"given":"Apostolos","family":"Tsiachristas","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0365-7775","authenticated-orcid":false,"given":"Rohan","family":"Borschmann","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3100-3234","authenticated-orcid":false,"given":"Navneet","family":"Kapur","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-3073-7619","authenticated-orcid":false,"given":"Galit","family":"Geulayov","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,6,2]]},"reference":[{"key":"R1","unstructured":"Self-harm: assessment, management and preventing recurrence. National Institute for Health and Care Excellence (NICE). 2022. URL: https:\/\/www.nice.org.uk\/guidance\/ng225 [Accessed 26-03-2026]"},{"key":"R2","unstructured":"Suicides in England and Wales. Office for National Statistics. 2024. URL: https:\/\/www.ons.gov.uk\/peoplepopulationandcommunity\/birthsdeathsandmarriages\/deaths\/datasets\/suicidesintheunitedkingdomreferencetables [Accessed 26-03-2026]"},{"key":"R3","doi-asserted-by":"publisher","DOI":"10.1017\/S2045796020000189","article-title":"Incidence and general hospital costs of self-harm across England: estimates based on the multicentre study of self-harm","volume":"29","author":"Tsiachristas","journal-title":"Epidemiol Psychiatr Sci"},{"issue":"2","key":"R4","doi-asserted-by":"publisher","first-page":"167","DOI":"10.1016\/S2215-0366(17)30478-9","article-title":"Incidence of suicide, hospital-presenting non-fatal self-harm, and community-occurring non-fatal self-harm in adolescents in England (the iceberg model of self-harm): a retrospective study","volume":"5","author":"Geulayov","journal-title":"Lancet Psychiatry"},{"issue":"9853","key":"R5","doi-asserted-by":"publisher","first-page":"1568","DOI":"10.1016\/S0140-6736(12)61141-6","article-title":"Premature death after self-harm: a multicentre cohort study","volume":"380","author":"Bergen","journal-title":"Lancet"},{"issue":"2","key":"R6","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0089944","article-title":"Hospital presenting self-harm and risk of fatal and non-fatal repetition: systematic review and meta-analysis","volume":"9","author":"Carroll","journal-title":"PLoS ONE"},{"issue":"12","key":"R7","doi-asserted-by":"publisher","first-page":"1021","DOI":"10.1016\/S2215-0366(19)30402-X","article-title":"Suicide following presentation to hospital for non-fatal self-harm in the multicentre study of self-harm: a long-term follow-up study","volume":"6","author":"Geulayov","journal-title":"Lancet Psychiatry"},{"key":"R8","doi-asserted-by":"publisher","DOI":"10.1136\/bmj.g5954","article-title":"Clinical and social outcomes of adolescent self harm: population based birth cohort study","volume":"349","author":"Mars","journal-title":"BMJ"},{"issue":"3","key":"R9","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1016\/S2352-4642(17)30007-X","article-title":"20-year outcomes in adolescents who self-harm: a population-based cohort study","volume":"1","author":"Borschmann","journal-title":"Lancet Child Adolesc Health"},{"issue":"2","key":"R10","doi-asserted-by":"publisher","DOI":"10.1136\/bmjopen-2015-009749","article-title":"Rates of self-harm presenting to general hospitals: a comparison of data from the multicentre study of self-harm in England and hospital episode statistics","volume":"6","author":"Clements","journal-title":"BMJ Open"},{"key":"R11","unstructured":"Hospital episode statistics (HES). NHS, England Digital. URL: https:\/\/digital.nhs.uk\/data-and-information\/data-tools-and-services\/data-services\/hospital-episode-statistics [Accessed 26-03-2026]"},{"issue":"4","key":"R12","doi-asserted-by":"publisher","first-page":"517","DOI":"10.1007\/s11739-015-1213-9","article-title":"Use of administrative data in healthcare research","volume":"10","author":"Mazzali","journal-title":"Intern Emerg Med"},{"issue":"4","key":"R13","doi-asserted-by":"publisher","first-page":"323","DOI":"10.1016\/j.jclinepi.2004.10.012","article-title":"A review of uses of health care utilization databases for epidemiologic research on therapeutics","volume":"58","author":"Schneeweiss","journal-title":"J Clin Epidemiol"},{"issue":"1","key":"R14","doi-asserted-by":"publisher","DOI":"10.1038\/s41598-018-25773-2","article-title":"Identifying suicide ideation and suicidal attempts in a psychiatric clinical research database using natural language processing","volume":"8","author":"Fernandes","journal-title":"Sci Rep"},{"issue":"8","key":"R15","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0253809","article-title":"Developing a natural language processing tool to identify perinatal self-harm in electronic healthcare records","volume":"16","author":"Ayre","journal-title":"PLoS ONE"},{"issue":"1","key":"R16","doi-asserted-by":"publisher","first-page":"9","DOI":"10.1177\/1403494817746274","article-title":"Contact with primary and mental health care prior to suicide: a systematic review of the literature from 2000 to 2017","volume":"47","author":"Stene-Larsen","journal-title":"Scand J Public Health"},{"key":"R17","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijmedinf.2023.105122","article-title":"Clinical named entity recognition and relation extraction using natural language processing of medical free text: a systematic review","volume":"177","author":"Fraile Navarro","journal-title":"Int J Med Inform"},{"key":"R18","doi-asserted-by":"crossref","unstructured":"Agrawal M Hegselmann S Lang H Kim Y Sontag D . Large language models are few-shot clinical information extractors. Presented at: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing; Dec 7-11, 2022; Abu Dhabi, United Arab Emirates. [doi: 10.18653\/v1\/2022.emnlp-main.130]","DOI":"10.18653\/v1\/2022.emnlp-main.130"},{"issue":"6","key":"R19","doi-asserted-by":"publisher","DOI":"10.1093\/bib\/bbac409","article-title":"BioGPT: generative pre-trained transformer for biomedical text generation and mining","volume":"23","author":"Luo","journal-title":"Brief Bioinformatics"},{"key":"R20","doi-asserted-by":"crossref","unstructured":"Liu J Chen Y Liu K Bi W Liu X . Event extraction as machine reading comprehension. Presented at: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP); Nov 16-20, 2020:1641-1651; Online. 2020.[doi: 10.18653\/v1\/2020.emnlp-main.128]","DOI":"10.18653\/v1\/2020.emnlp-main.128"},{"issue":"1","key":"R21","doi-asserted-by":"publisher","DOI":"10.1007\/s13278-025-01428-9","article-title":"Large language models (LLM) in computational social science: prospects, current state, and challenges","volume":"15","author":"Thapa","journal-title":"Soc Netw Anal Min"},{"key":"R22","unstructured":"Dubey A Jauhri A Pandey A Kadian A Al-Dahle A Letman A et al. The Llama 3 herd of models. arXiv. Preprint posted online on Jul 31, 2024. [doi: 10.48550\/arXiv.2407.21783]"},{"key":"R23","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2025.110246","article-title":"Detecting the clinical features of difficult-to-treat depression using synthetic data from large language models","volume":"194","author":"Lorge","journal-title":"Comput Biol Med"},{"issue":"1","key":"R24","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01233-2","article-title":"Privacy-preserving large language models for structured medical information retrieval","volume":"7","author":"Wiest","journal-title":"NPJ Digit Med"},{"issue":"6","key":"R25","doi-asserted-by":"publisher","first-page":"532","DOI":"10.1192\/bjp.2024.134","article-title":"Detection of suicidality from medical text using privacy-preserving large language models","volume":"225","author":"Wiest","journal-title":"Br J Psychiatry"},{"issue":"7","key":"R26","doi-asserted-by":"publisher","first-page":"513","DOI":"10.1007\/s00127-007-0199-7","article-title":"Self-harm in England: a tale of three cities","volume":"42","author":"Hawton","journal-title":"Soc Psychiat Epidemiol"},{"key":"R27","unstructured":"Your strategic partner for neuroscience, dementia, neurology & CNS -innovation. Akrivia Health. URL: https:\/\/akriviahealth.com\/ [Accessed 26-03-2026]"},{"key":"R28","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2020.103429","article-title":"Maximizing the use of social and behavioural information from secondary care mental health electronic health records","volume":"107","author":"Goodday","journal-title":"J Biomed Inform"},{"key":"R29","doi-asserted-by":"crossref","unstructured":"Wei J Wang X Schuurmans D Bosma M Ichter B Xia F et al. Chain-of-thought prompting elicits reasoning in large language models. Presented at: 36th Conference on Neural Information Processing Systems (NeurIPS 2022); Nov 28 to Dec 9, 2022; New Orleans, Louisiana, United States. 2022.[doi: 10.52202\/068431-1800]","DOI":"10.52202\/068431-1800"},{"key":"R30","doi-asserted-by":"crossref","unstructured":"Huang J Chang KCC . Towards reasoning in large language models: a survey. Presented at: Findings of the Association for Computational Linguistics; Jul 9-14, 2023; Toronto, Canada. [doi: 10.18653\/v1\/2023.findings-acl.67]","DOI":"10.18653\/v1\/2023.findings-acl.67"},{"issue":"5","key":"R31","doi-asserted-by":"publisher","DOI":"10.1016\/j.isci.2024.109713","article-title":"The application of large language models in medicine: a scoping review","volume":"27","author":"Meng","journal-title":"iScience"},{"key":"R32","unstructured":"Gemma TeamAishwarya Kamath JF Pathak S et al. Gemma 3 technical report. arXiv. Preprint posted online on Mar 25, 2025. [doi: 10.48550\/arXiv.2503.19786]"},{"key":"R33","unstructured":"Ollama (version 0.11.11). Ollama. URL: https:\/\/ollama.com\/ [Accessed 02-04-2026]"},{"key":"R34","unstructured":"Paszke A Gross S Massa F Lerer A Bradbury J Chanan G et al. Pytorch: an imperative style, high-performance deep learning library. arXiv. Preprint posted online on Dec 3, 2019. [doi: 10.48550\/arXiv.1912.01703]"},{"key":"R35","doi-asserted-by":"crossref","unstructured":"Reynolds L McDonell K . Prompt programming for large language models: beyond the few-shot paradigm. Presented at: CHI \u201921; 1-7; Yokohama Japan. May 8, 2021.[doi: 10.1145\/3411763.3451760]","DOI":"10.1145\/3411763.3451760"},{"issue":"9","key":"R36","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3560815","article-title":"Pre-train, prompt, and predict: a systematic survey of prompting methods in natural language processing","volume":"55","author":"Liu","journal-title":"ACM Comput Surv"},{"key":"R37","unstructured":"CRIS privacy notice. Oxford Health NHS Foundation Trust. URL: https:\/\/www.oxfordhealth.nhs.uk\/research\/toolkit\/cris\/cris-privacy-notice [Accessed 26-03-2026]"},{"key":"R38","doi-asserted-by":"crossref","unstructured":"Altman DG . Practical Statistics for Medical Research. Chapman and Hall\/CRC; 1990. [doi: 10.1201\/9780429258589]","DOI":"10.1201\/9780429258589"},{"issue":"10461","key":"R39","doi-asserted-by":"publisher","first-page":"1445","DOI":"10.1016\/S0140-6736(24)01121-8","article-title":"The Lancet Commission on self-harm","volume":"404","author":"Moran","journal-title":"Lancet"},{"key":"R40","unstructured":"Kaplan J McCandlish S Henighan T et al. Scaling laws for neural language models. arXiv. Preprint posted online on Jan 23, 2020. [doi: 10.48550\/arXiv.2001.08361]"},{"key":"R41","unstructured":"Hoffmann J . Training compute-optimal large language models. arXiv. Preprint posted online on Mar 29, 2022. [doi: 10.48550\/arXiv.2203.15556]"},{"key":"R42","doi-asserted-by":"crossref","unstructured":"Lester B Al-Rfou R Constant N . The power of scale for parameter-efficient prompt tuning. Presented at: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing; Nov 7-11, 2021:3045-3059; Online and Punta Cana, Dominican Republic. [doi: 10.18653\/v1\/2021.emnlp-main.243]","DOI":"10.18653\/v1\/2021.emnlp-main.243"}],"container-title":["JMIR Mental Health"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/mental.jmir.org\/2026\/1\/e87586","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,31]],"date-time":"2026-07-31T22:25:10Z","timestamp":1785536710000},"score":1,"resource":{"primary":{"URL":"https:\/\/mental.jmir.org\/2026\/1\/e87586"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,2]]},"references-count":42,"URL":"https:\/\/doi.org\/10.2196\/87586","relation":{},"ISSN":["2368-7959"],"issn-type":[{"value":"2368-7959","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,2]]},"article-number":"v13i2e87586"}}