{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,22]],"date-time":"2026-07-22T03:45:04Z","timestamp":1784691904330,"version":"3.55.0"},"reference-count":21,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR AI"],"abstract":"\n Background<\/jats:title>\n People with schizophrenia often present with cognitive impairments that may hinder their ability to learn about their condition. Education platforms powered by large language models (LLMs) have the potential to improve the accessibility of mental health information. However, the black-box nature of LLMs raises ethical and safety concerns regarding the controllability of chatbots. In particular, prompt-engineered chatbots may drift from their intended role as the conversation progresses and become more prone to hallucinations.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n This study aimed to develop and evaluate a critical analysis filter (CAF) system that ensures that an LLM-powered prompt-engineered chatbot reliably complies with its predefined instructions and scope while delivering validated mental health information.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n For a proof of concept, we prompt engineered an educational chatbot for schizophrenia powered by GPT-4 that could dynamically access information from a schizophrenia manual written for people with schizophrenia and their caregivers. In the CAF, a team of prompt-engineered LLM agents was used to critically analyze and refine the chatbot\u2019s responses and deliver real-time feedback to the chatbot. To assess the ability of the CAF to re-establish the chatbot\u2019s adherence to its instructions, we generated 3 conversations (by conversing with the chatbot with the CAF disabled) wherein the chatbot started to drift from its instructions toward various unintended roles. We used these checkpoint conversations to initialize automated conversations between the chatbot and adversarial chatbots designed to entice it toward unintended roles. Conversations were repeatedly sampled with the CAF enabled and disabled. In total, 3 human raters independently rated each chatbot response according to criteria developed to measure the chatbot\u2019s integrity, specifically, its transparency (such as admitting when a statement lacked explicit support from its scripted sources) and its tendency to faithfully convey the scripted information in the schizophrenia manual.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n In total, 36 responses (3 different checkpoint conversations, 3 conversations per checkpoint, and 4 adversarial queries per conversation) were rated for compliance with the CAF enabled and disabled. Activating the CAF resulted in a compliance score that was considered acceptable (\u22652) in 81% (7\/36) of the responses, compared to only 8.3% (3\/36) when the CAF was deactivated.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Although more rigorous testing in realistic scenarios is needed, our results suggest that self-reflection mechanisms could enable LLMs to be used effectively and safely in educational mental health platforms. This approach harnesses the flexibility of LLMs while reliably constraining their scope to appropriate and accurate interactions.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/69820","type":"journal-article","created":{"date-parts":[[2025,2,24]],"date-time":"2025-02-24T12:05:07Z","timestamp":1740398707000},"page":"e69820","source":"Crossref","is-referenced-by-count":9,"title":["Prompt Engineering an Informational Chatbot for Education on Mental Health Using a Multiagent Approach for Enhanced Compliance With Prompt Instructions: Algorithm Development and Validation"],"prefix":"10.2196","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0370-4463","authenticated-orcid":false,"given":"Per Niklas","family":"Waaler","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4494-1593","authenticated-orcid":false,"given":"Musarrat","family":"Hussain","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-7550-2913","authenticated-orcid":false,"given":"Igor","family":"Molchanov","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7544-2482","authenticated-orcid":false,"given":"Lars Ailo","family":"Bongo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0441-3729","authenticated-orcid":false,"given":"Brita","family":"Elvev\u00e5g","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2025,3,26]]},"reference":[{"key":"ref1","doi-asserted-by":"publisher","DOI":"10.1001\/jamapsychiatry.2023.1253"},{"key":"ref2","author":"Xu, H","journal-title":"arXiv"},{"key":"ref3","doi-asserted-by":"publisher","DOI":"10.1038\/s41380-023-01949-9"},{"key":"ref4","doi-asserted-by":"publisher","DOI":"10.2196\/17412"},{"key":"ref5","doi-asserted-by":"publisher","DOI":"10.2196\/53672"},{"key":"ref6","doi-asserted-by":"publisher","DOI":"10.2196\/54781"},{"key":"ref7","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.acl-long.546"},{"key":"ref8","doi-asserted-by":"publisher","DOI":"10.21203\/rs.3.rs-2566942\/v1"},{"key":"ref9","doi-asserted-by":"publisher","DOI":"10.2196\/16222"},{"key":"ref10","author":"Sahoo, P","journal-title":"arXiv"},{"key":"ref11","unstructured":"Learning to live with schizophrenia: a companion guideGAMIAN Europe20222024-04-29https:\/\/www.gamian.eu\/wp-content\/uploads\/Gamian-Schizophrenia-Guide-2016.pdf"},{"key":"ref12","unstructured":"TRUSTING aims to develop a user- friendly, trustworthy speech-based tool for the prediction of relapse in psychosisTrusting Project2025-01-30https:\/\/trusting-project.eu\/"},{"key":"ref13","doi-asserted-by":"publisher","DOI":"10.1145\/3491102.3517582"},{"key":"ref14","doi-asserted-by":"publisher","DOI":"10.2196\/60807"},{"key":"ref15","doi-asserted-by":"publisher","DOI":"10.2196\/54571"},{"key":"ref16","unstructured":"Welcome to GAMIAN-Europe: global alliance of mental illness advocacy networkGAMIAN-Europe2024-09-10https:\/\/www.gamian.eu\/"},{"key":"ref17","doi-asserted-by":"publisher","DOI":"10.1016\/S2589-7500(22)00153-4"},{"key":"ref18","author":"Asai, A","journal-title":"arXiv"},{"key":"ref19","author":"Lewis, P","journal-title":"arXiv"},{"key":"ref20","doi-asserted-by":"publisher","DOI":"10.1001\/jamaoncol.2024.0836"},{"key":"ref21","unstructured":"uit-hdl\/chatbot-for-mental-healthGitHub2024-08-31https:\/\/github.com\/uit-hdl\/chatbot-for-mental-health"}],"container-title":["JMIR AI"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/ai.jmir.org\/2025\/1\/e69820","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T17:58:00Z","timestamp":1782842280000},"score":1,"resource":{"primary":{"URL":"https:\/\/ai.jmir.org\/2025\/1\/e69820"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,3,26]]},"references-count":21,"URL":"https:\/\/doi.org\/10.2196\/69820","relation":{"has-preprint":[{"id-type":"doi","id":"10.2196\/preprints.69820","asserted-by":"object"}]},"ISSN":["2817-1705"],"issn-type":[{"value":"2817-1705","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,3,26]]}}}