{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,15]],"date-time":"2026-06-15T08:21:44Z","timestamp":1781511704132,"version":"3.54.1"},"reference-count":142,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,10,12]],"date-time":"2023-10-12T00:00:00Z","timestamp":1697068800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Artif. Intell."],"abstract":"<jats:p>Virtual Mental Health Assistants (VMHAs) continuously evolve to support the overloaded global healthcare system, which receives approximately 60 million primary care visits and 6 million emergency room visits annually. These systems, developed by clinical psychologists, psychiatrists, and AI researchers, are designed to aid in Cognitive Behavioral Therapy (CBT). The main focus of VMHAs is to provide relevant information to mental health professionals (MHPs) and engage in meaningful conversations to support individuals with mental health conditions. However, certain gaps prevent VMHAs from fully delivering on their promise during active communications. One of the gaps is their inability to explain their decisions to patients and MHPs, making conversations less trustworthy. Additionally, VMHAs can be vulnerable in providing unsafe responses to patient queries, further undermining their reliability. In this review, we assess the current state of VMHAs on the grounds of user-level explainability and safety, a set of desired properties for the broader adoption of VMHAs. This includes the examination of ChatGPT, a conversation agent developed on AI-driven models: GPT3.5 and GPT-4, that has been proposed for use in providing mental health services. By harnessing the collaborative and impactful contributions of AI, natural language processing, and the mental health professionals (MHPs) community, the review identifies opportunities for technological progress in VMHAs to ensure their capabilities include explainable and safe behaviors. It also emphasizes the importance of measures to guarantee that these advancements align with the promise of fostering trustworthy conversations.<\/jats:p>","DOI":"10.3389\/frai.2023.1229805","type":"journal-article","created":{"date-parts":[[2023,10,12]],"date-time":"2023-10-12T12:46:21Z","timestamp":1697114781000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":34,"title":["A review of the explainability and safety of conversational agents for mental health to identify avenues for improvement"],"prefix":"10.3389","volume":"6","author":[{"given":"Surjodeep","family":"Sarkar","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Manas","family":"Gaur","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lujie Karen","family":"Chen","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Muskan","family":"Garg","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Biplav","family":"Srivastava","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1965","published-online":{"date-parts":[[2023,10,12]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"e17828","DOI":"10.2196\/17828","article-title":"Perceptions and opinions of patients about mental health chatbots: scoping review","volume":"23","author":"Abd-Alrazaq","year":"2021","journal-title":"J. Med. Internet Res"},{"key":"B2","doi-asserted-by":"publisher","first-page":"923","DOI":"10.1007\/s10796-022-10254-9","article-title":"Designing personality-adaptive conversational agents for mental health care","volume":"24","author":"Ahmad","year":"2022","journal-title":"Inf. Syst. Front"},{"key":"B3","doi-asserted-by":"publisher","first-page":"463","DOI":"10.1162\/tacl_a_00111","article-title":"Large-scale analysis of counseling conversations: an application of natural language processing to mental health","volume":"4","author":"Althoff","year":"2016","journal-title":"Trans. Assoc. Comput. Linguist"},{"key":"B4","article-title":"Training a helpful and harmless assistant with reinforcement learning from human feedback","author":"Bai","year":"","journal-title":"arXiv preprint arXiv"},{"key":"B5","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2212.08073","article-title":"Constitutional ai: harmlessness from ai feedback","author":"Bai","year":"","journal-title":"arXiv [Preprint]. arXiv:2212.08073"},{"key":"B6","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2212.1001","article-title":"Docasref: A pilot empirical study on repurposing reference-based summary quality metrics reference-freely","author":"Bao","year":"2022","journal-title":"arXiv [Preprint]. arXiv:2212.10013"},{"key":"B7","doi-asserted-by":"crossref","DOI":"10.1145\/3351095.3375624","article-title":"\u201cExplainable machine learning in deployment,\u201d","volume-title":"Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency","author":"Bhatt","year":"2020"},{"key":"B8","doi-asserted-by":"publisher","first-page":"D267","DOI":"10.1093\/nar\/gkh061","article-title":"The unified medical language system (umls): integrating biomedical terminology","volume":"32","author":"Bodenreider","year":"2004","journal-title":"Nucleic Acids Res"},{"key":"B9","article-title":"On the opportunities and risks of foundation models","author":"Bommasani","year":"2021","journal-title":"arXiv"},{"key":"B10","doi-asserted-by":"crossref","first-page":"395","DOI":"10.1109\/BigComp57234.2023.00097","article-title":"\u201cDeep learning mental health dialogue system,\u201d","author":"Brocki","year":"2023","journal-title":"2023 IEEE International Conference on Big Data and Smart Computing (BigComp)"},{"key":"B11","doi-asserted-by":"publisher","first-page":"109","DOI":"10.2165\/00002018-199920020-00002","article-title":"The medical dictionary for regulatory activities (meddra)","volume":"20","author":"Brown","year":"1999","journal-title":"Drug Safety"},{"key":"B12","unstructured":"CampbellD. Strain on Mental Health Care Leaves 8m People Without Help, Say NHS Leaders2021"},{"key":"B13","doi-asserted-by":"publisher","first-page":"87","DOI":"10.1109\/TMM.2020.3046867","article-title":"Building and using personal knowledge graph to improve suicidal ideation detection on social media","volume":"24","author":"Cao","year":"2020","journal-title":"IEEE Trans. Multimed"},{"key":"B14","doi-asserted-by":"publisher","first-page":"339","DOI":"10.1111\/1468-5973.12319","article-title":"Ai-powered mental health chatbots: Examining users? motivations, active communicative action and engagement after mass-shooting disasters","volume":"28","author":"Cheng","year":"2020","journal-title":"J. Conting. Crisis Manage"},{"key":"B15","doi-asserted-by":"publisher","first-page":"e18607","DOI":"10.2196\/18607","article-title":"A chatbot for perinatal women's and partners? obstetric and mental health care: development and usability evaluation study","volume":"9","author":"Chung","year":"2021","journal-title":"JMIR Medical Informatics"},{"key":"B16","doi-asserted-by":"publisher","first-page":"81","DOI":"10.1038\/s41746-020-0288-5","article-title":"Sex and gender differences and biases in artificial intelligence for biomedicine and healthcare","volume":"3","author":"Cirillo","year":"2020","journal-title":"NPJ Digital Med"},{"key":"B17","first-page":"2376","article-title":"\u201cA study of automatic metrics for the evaluation of natural language explanations\u201d","author":"Clinciu","year":"2021","journal-title":"Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume"},{"key":"B18","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2304.11111","article-title":"Inducing anxiety in large language models increases exploration and bias","author":"Coda-Forno","year":"2023","journal-title":"arXiv [Preprint]. arXiv:2304.11111"},{"key":"B19","doi-asserted-by":"publisher","first-page":"20552076231183542","DOI":"10.1177\/20552076231183542","article-title":"To chat or bot to chat: Ethical issues with using chatbots in mental health","volume":"9","author":"Coghlan","year":"2023","journal-title":"Digital Health"},{"key":"B20","unstructured":"Cogito:Emotion and Conversation AI2011"},{"key":"B21","volume-title":"Mental Health Answers from Counselors","year":"2015"},{"key":"B22","doi-asserted-by":"publisher","first-page":"1049","DOI":"10.15585\/mmwr.mm6932a1","article-title":"Mental health, substance use, and suicidal ideation during the covid-19 pandemic?United States, June 24\u201330, 2020","volume":"69","author":"Czeisler","year":"2020","journal-title":"Morbid. Mortal. Wkly. Rep"},{"key":"B23","doi-asserted-by":"publisher","first-page":"9137","DOI":"10.3390\/ijerph17239137","article-title":"A process evaluation examining the performance, adherence, and acceptability of a physical activity and diet artificial intelligence virtual health assistant","volume":"17","author":"Davis","year":"2020","journal-title":"Int. J. Environ. Res. Public Health"},{"key":"B24","unstructured":"DawsR. Babylon Health Lashes Out At Doctor Who Raised AI Chatbot Safety Concerns2020"},{"key":"B25","doi-asserted-by":"crossref","first-page":"1","DOI":"10.18653\/v1\/W19-3001","article-title":"\u201cTowards augmenting crisis counselor training by improving message retrieval,\u201d","volume-title":"Proceedings of the Sixth Workshop on Computational Linguistics and Clinical Psychology","author":"Demasi","year":"2019"},{"key":"B26","doi-asserted-by":"publisher","first-page":"1170","DOI":"10.1109\/TETC.2020.2974478","article-title":"A mental health chatbot for regulating emotions (sermo)-concept and usability test","volume":"9","author":"Denecke","year":"2020","journal-title":"IEEE Trans. Emerg. Topics Comput"},{"key":"B27","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2022.acl-long.284","article-title":"\u201cSafetykit: First aid for measuring safety in open-domain conversational systems,\u201d","volume-title":"Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Dinan","year":"2022"},{"key":"B28","unstructured":"DinanR. 1st Safety for Conversational AI Workshop | ACL Member Portal2020"},{"key":"B29","article-title":"Anticipating safety issues in e2e conversational AI: framework and tooling","author":"Dinan","year":"2021","journal-title":"arXiv [Preprint].arXiv:2107.03451"},{"key":"B30","article-title":"\u201cWizard of Wikipedia: knowledge-powered conversational agents,\u201d","author":"Dinan","year":"2018","journal-title":"International Conference on Learning Representations (Kigali)"},{"key":"B31","first-page":"279","article-title":"Snomed-ct: the advanced terminology and coding system for ehealth","volume":"121","author":"Donnelly","year":"2006","journal-title":"Stud. Health Technol. Inform"},{"key":"B32","unstructured":"DugganK. Z. Limbic Mental Health E-Triage Chatbot Gets UKCA Certification1972"},{"key":"B33","article-title":"A conversational interface to improve medication adherence: towards AI support in patient's treatment","author":"Fadhil","year":"2018","journal-title":"arXiv [Preprint]. arXiv:1803.09844"},{"key":"B34","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1002\/9781118625392.wbecp351","article-title":"Structured clinical interview for the dsm (scid)","volume":"351","author":"First","year":"2014","journal-title":"Encyclop. Clini. Psychol"},{"key":"B35","doi-asserted-by":"publisher","first-page":"e7785","DOI":"10.2196\/mental.7785","article-title":"Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (woebot): a randomized controlled trial","volume":"4","author":"Fitzpatrick","year":"2017","journal-title":"JMIR Mental Health"},{"key":"B36","doi-asserted-by":"publisher","first-page":"681","DOI":"10.1007\/s11023-020-09548-1","article-title":"Gpt-3: Its nature, scope, limits, and consequences","volume":"30","author":"Floridi","year":"2020","journal-title":"Minds Mach"},{"key":"B37","article-title":"Evaluating superhuman models with consistency checks","author":"Fluri","year":"2023","journal-title":"arXiv [Preprint]. arXiv:2306.09983"},{"key":"B38","doi-asserted-by":"publisher","first-page":"e9782","DOI":"10.2196\/preprints.9782","article-title":"Using psychological artificial intelligence (tess) to relieve symptoms of depression and anxiety: randomized controlled trial","volume":"5","author":"Fulmer","year":"2018","journal-title":"JMIR Mental Health"},{"key":"B39","doi-asserted-by":"publisher","first-page":"5","DOI":"10.1109\/MIC.2022.3179759","article-title":"Knowledge-infused learning: a sweet spot in neuro-symbolic ai","volume":"26","author":"Gaur","year":"","journal-title":"IEEE Inter. Comp"},{"key":"B40","doi-asserted-by":"publisher","first-page":"10672","DOI":"10.1609\/aaai.v36i10.21312","article-title":"Iseeq: Information seeking question generation using dynamic meta-information retrieval and knowledge graphs","volume":"36","author":"Gaur","year":"","journal-title":"Proc. Innov. Appl. Artif. Intell. Conf"},{"key":"B41","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/ACII.2019.8925455","article-title":"\u201cEmma: An emotion-aware wellbeing chatbot,\u201d","volume-title":"2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII)","author":"Ghandeharioun","year":"2019"},{"key":"B42","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2022\/704","article-title":"Am i no good? Towards detecting perceived burdensomeness and thwarted belongingness from suicide notes","author":"Ghosh","year":"2022","journal-title":"arXiv [Preprint]. arXiv:2206.06141."},{"key":"B43","unstructured":"In-the-Moment Care for Every Emotion2011"},{"key":"B44","unstructured":"GrigorutaC. Why We Need Mental Health Chatbots2018"},{"key":"B45","volume-title":"Deepmind Introduces 'Sparrow,' An Artificial Intelligence-Powered Chatbot Developed to Build Safer Machine Learning Systems","year":"2022"},{"key":"B46","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.clpsych-1.12","article-title":"Learning to automate follow-up question generation using process knowledge for depression triage on reddit posts","author":"Gupta","year":"2022","journal-title":"arXiv"},{"key":"B47","doi-asserted-by":"publisher","first-page":"9712","DOI":"10.3390\/app12199712","article-title":"Interdisciplinary iot and emotion knowledge graph-based recommendation system to boost mental health","volume":"12","author":"Gyrard","year":"2022","journal-title":"Appl. Sci"},{"key":"B48","unstructured":"Limbic Access AI Conversational Chatbot for e-triage - 2023"},{"key":"B49","unstructured":"HarrisonC. GitHub - 2023"},{"key":"B50","doi-asserted-by":"publisher","first-page":"e11671","DOI":"10.2196\/11671","article-title":"Utilization of patient-generated data collected through mobile devices: insights from a survey on attitudes toward mobile self-monitoring and self-management apps for depression","volume":"6","author":"Hartmann","year":"2019","journal-title":"JMIR Mental Health"},{"key":"B51","doi-asserted-by":"publisher","DOI":"10.1145\/3278721.3278777","article-title":"\u201cEthical challenges in data-driven dialogue systems,\u201d","author":"Henderson","year":"2018","journal-title":"Proceedings of the"},{"key":"B52","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2306.12001","article-title":"An overview of catastrophic ai risks","author":"Hendrycks","year":"2023","journal-title":"arXiv"},{"key":"B53","doi-asserted-by":"publisher","first-page":"e32832","DOI":"10.2196\/32832","article-title":"Diagnostic performance of an app-based symptom checker in mental disorders: comparative study in psychotherapy outpatients","volume":"9","author":"Hennemann","year":"2022","journal-title":"JMIR Ment Health"},{"key":"B54","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1812.04608","article-title":"Metrics for explainable AI: challenges and prospects","author":"Hoffman","year":"2018","journal-title":"arXiv [Preprint]. arXiv:1812.04608"},{"key":"B55","volume-title":"Language Use in Teenage Crisis Intervention and the Immediate Outcome: A Machine Automated Analysis of Large Scale Text Data","author":"Huang","year":"2015"},{"key":"B56","volume-title":"Self-Disclosure and its Impact on Individuals Who Receive Mental Health Services (hhs pub. no. sma-08-4337)","author":"Hyman","year":"2008"},{"key":"B57","unstructured":"What You Need to Know About Replika2022"},{"key":"B58","doi-asserted-by":"publisher","first-page":"e12106","DOI":"10.2196\/12106","article-title":"An empathy-driven, conversational artificial intelligence agent (wysa) for digital mental well-being: real-world data evaluation mixed-methods study","volume":"6","author":"Inkster","year":"2018","journal-title":"JMIR mHealth"},{"key":"B59","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1038\/s41746-023-00751-9","article-title":"Explainable artificial intelligence for mental health through transparency and interpretability for understandability","volume":"6","author":"Joyce","year":"2023","journal-title":"NPJ Digital Med"},{"key":"B60","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2004.14667","article-title":"NUBIA: neural based interchangeability assessor for text generation","author":"Kane","year":"2020","journal-title":"arXiv [Preprint]. arXiv: 2004.14667"},{"key":"B61","doi-asserted-by":"publisher","first-page":"1","DOI":"10.5120\/8844-2886","article-title":"Medchatbot: An umls based chatbot for medical students","volume":"55","author":"Kazi","year":"2012","journal-title":"Int. J. Comp. Appl"},{"key":"B62","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.emnlp-main.267","article-title":"\u201cProsocialdialog: A prosocial backbone for conversational agents,\u201d","author":"Kim","year":"2022","journal-title":"Proceedings of the"},{"key":"B63","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P18-1249","article-title":"\u201cConstituency parsing with a self-attentive encoder,\u201d","author":"Kitaev","year":"2018","journal-title":"Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"},{"key":"B64","doi-asserted-by":"publisher","first-page":"100373","DOI":"10.1016\/j.simpa.2022.100373","article-title":"Accurate clinical and biomedical named entity recognition at scale","volume":"13","author":"Kocaman","year":"2022","journal-title":"Softw. Impac"},{"key":"B65","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3485874","article-title":"Chatbots to support young adults? mental health: an exploratory study of acceptability","volume":"12","author":"Koulouri","year":"2022","journal-title":"ACM Trans. Interact. Intell. Syst"},{"key":"B66","doi-asserted-by":"publisher","first-page":"e829?e840","DOI":"10.1016\/S2589-7500(22)00153-4","article-title":"From promise to practice: towards the realisation of ai-informed mental health care","volume":"4","author":"Koutsouleris","year":"2022","journal-title":"Lancet Digital Health"},{"key":"B67","doi-asserted-by":"publisher","first-page":"606","DOI":"10.1046\/j.1525-1497.2001.016009606.x","article-title":"The phq-9: validity of a brief depression severity measure","volume":"16","author":"Kroenke","year":"2001","journal-title":"J. Gen. Intern. Med"},{"key":"B68","volume-title":"Self-Injury Support Online: Exploring Use of the Mobile Peer Support Application TalkLife","author":"Kruzan","year":"2019"},{"key":"B69","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.findings-naacl.67","article-title":"\u201cLearning rich representation of keyphrases from text,\u201d","author":"Kulkarni","year":"2022","journal-title":"Findings of the Association for Computational Linguistics: NAACL"},{"key":"B70","first-page":"3723","article-title":"\u201cFunctional transparency for structured data: a game-theoretic approach,\u201d","volume-title":"International Conference on Machine Learning","author":"Lee","year":"2019"},{"key":"B71","first-page":"1","volume-title":"Restatement and question generation for counsellor chatbot. In 1st Workshop on Natural Language Processing for Programming (NLP4Prog)","author":"Lee","year":"2021"},{"key":"B72","author":"Leiter","year":"2023","journal-title":"ChatGPT: A Meta-Analysis After 2.5 Months"},{"key":"B73","doi-asserted-by":"crossref","first-page":"32","DOI":"10.18653\/v1\/2021.sigdial-1.5","article-title":"\u201cEvaluation of in-person counseling strategies to develop physical activity chatbot for women,\u201d","volume-title":"Proceedings of the 22nd Annual Meeting of the Special Interest Group on Discourse and Dialogue","author":"Liang","year":"2021"},{"key":"B74","unstructured":"Enabling the Best Psychological Therapy2017"},{"key":"B75","doi-asserted-by":"crossref","first-page":"1014","DOI":"10.18653\/v1\/P16-1096","article-title":"\u201cNormalising medical concepts in social media texts by learning semantic representation,\u201d","author":"Limsopatham","year":"2016","journal-title":"Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (volume 1: long papers)"},{"key":"B76","first-page":"74","article-title":"\u201cRouge: a package for automatic evaluation of summaries\u201d","volume-title":"Text Summarization Branches Out","author":"Lin","year":"2004"},{"key":"B77","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1016\/j.jairtraman.2015.12.007","article-title":"Service quality and price perception of service: influence on word-of-mouth and revisit intention","volume":"52","author":"Liu","year":"2016","journal-title":"J. Air Transport Manage"},{"key":"B78","doi-asserted-by":"publisher","first-page":"17","DOI":"10.1109\/MITP.2005.122","article-title":"Rxnorm: prescription for electronic drug information exchange","volume":"7","author":"Liu","year":"2005","journal-title":"IT Prof"},{"key":"B79","first-page":"873","article-title":"\u201cImproved image captioning via policy gradient optimization of spider,\u201d","author":"Liu","year":"2017","journal-title":"Proceedings of the IEEE International Conference on Computer Vision"},{"key":"B80","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-57321-8_1","article-title":"\u201cExplainable artificial intelligence: Concepts, applications, research challenges and visions,\u201d","volume-title":"Machine Learning and Knowledge Extraction","author":"Longo","year":"2020"},{"key":"B81","doi-asserted-by":"publisher","first-page":"30","DOI":"10.48550\/arXiv.1705.07874","article-title":"A unified approach to interpreting model predictions","volume":"2017","author":"Lundberg","year":"2017","journal-title":"Adv. Neural Inf. Process. Syst"},{"key":"B82","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2302.00871","article-title":"Using in-context learning to improve dialogue safety","author":"Meade","year":"2023","journal-title":"arXiv"},{"key":"B83","doi-asserted-by":"publisher","first-page":"825565","DOI":"10.3389\/frai.2022.825565","article-title":"Ganterfactual?counterfactual explanations for medical non-experts using generative adversarial learning","volume":"5","author":"Mertes","year":"2022","journal-title":"Front. Artif. Intell"},{"key":"B84","unstructured":"FAIR Principles - 2017"},{"key":"B85","doi-asserted-by":"publisher","DOI":"10.1145\/2974804.2974820","article-title":"\u201cConversational agents and mental health: theory-informed assessment of language and affect,\u201d","author":"Miner","year":"2016","journal-title":"Proceedings of the Fourth International Conference on Human Agent Interaction"},{"key":"B86","doi-asserted-by":"publisher","first-page":"e33717","DOI":"10.2196\/33717","article-title":"Developing, implementing, and evaluating an artificial intelligence\u2013guided mental health resource navigation chatbot for health care workers and their families during and following the COVID-19 pandemic: protocol for a cross-sectional study","volume":"11","author":"Noble","year":"2022","journal-title":"JMIR Res Protoc"},{"key":"B87","doi-asserted-by":"publisher","DOI":"10.3115\/1073083.1073135","article-title":"\u201cBleu: a method for automatic evaluation of machine translation,\u201d","author":"Papineni","year":"2002","journal-title":"Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics"},{"key":"B88","doi-asserted-by":"publisher","first-page":"3419","DOI":"10.18653\/v1\/2022.emnlp-main.225","article-title":"\u201cRed teaming language models with language models,\u201d","author":"Perez","year":"2022","journal-title":"Proceedings of the"},{"key":"B89","doi-asserted-by":"publisher","first-page":"18","DOI":"10.12746\/swrccc.v11i48.1193","article-title":"ChatGPT and medicine: Fears, fantasy, and the future of physicians","volume":"11","author":"Peterson","year":"2023","journal-title":"Southwest respir. Crit. Care chron"},{"key":"B90","first-page":"2008","volume-title":"Columbia-Suicide Severity Rating Scale (c-ssrs)","author":"Posner","year":"2008"},{"key":"B91","doi-asserted-by":"publisher","first-page":"1725","DOI":"10.1007\/s00146-021-01379-7","article-title":"Psychoanalyzing artificial intelligence: the case of replika","volume":"38","author":"Possati","year":"2022","journal-title":"AI Society"},{"key":"B92","doi-asserted-by":"publisher","first-page":"e16222","DOI":"10.2196\/16222","article-title":"Trust me, i'ma chatbot: how artificial intelligence in health care fails the turing test","volume":"21","author":"Powell","year":"2019","journal-title":"J. Med. Internet Res"},{"key":"B93","doi-asserted-by":"publisher","first-page":"4279","DOI":"10.24963\/ijcai.2018\/595","article-title":"Assigning personality\/profile to a chatting machine for coherent conversation generation","volume":"2018","author":"Qian","year":"2018","journal-title":"IJCAI"},{"key":"B94","doi-asserted-by":"publisher","first-page":"1130","DOI":"10.1097\/01.mlr.0000182534.19832.83","article-title":"Coding algorithms for defining comorbidities in icd-9-cm and icd-10 administrative data","volume":"43","author":"Quan","year":"2005","journal-title":"Med. Care"},{"key":"B95","unstructured":"Mental Health Care, Made Easier2014"},{"key":"B96","doi-asserted-by":"publisher","first-page":"137","DOI":"10.1007\/s11747-019-00710-5","article-title":"Explainable AI: from black box to glass box","volume":"48","author":"Rai","year":"2020","journal-title":"J. Acad. Market. Sci"},{"key":"B97","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1534","article-title":"Towards empathetic open-domain conversation models: a new benchmark and dataset","author":"Rashkin","year":"2018","journal-title":"arXiv"},{"key":"B98","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1186\/s12859-022-04751-6","article-title":"Coquad: a covid-19 question answering dataset system, facilitating research, benchmarking, and practice","volume":"23","author":"Raza","year":"2022","journal-title":"BMC Bioinformat"},{"key":"B99","doi-asserted-by":"publisher","first-page":"92","DOI":"10.1002\/wps.20050","article-title":"The dsm-5: classification and criteria changes","volume":"12","author":"Regier","year":"2013","journal-title":"World Psychiat"},{"key":"B100","doi-asserted-by":"publisher","first-page":"1135","DOI":"10.1145\/2939672.2939778","article-title":"\u201cWhy should i trust you? Explaining the predictions of any classifier,\u201d","author":"Ribeiro","year":"2016","journal-title":"Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining"},{"key":"B101","doi-asserted-by":"publisher","first-page":"2022","DOI":"10.1101\/2022.11.03.22281887","article-title":"Conversational ai facilitates mental health assessments and is associated with improved recovery rates","volume":"2022","author":"Rollwage","year":"2022","journal-title":"medRxiv"},{"key":"B102","doi-asserted-by":"crossref","first-page":"1586","DOI":"10.18653\/v1\/D18-1187","article-title":"\u201cLessons from natural language inference in the clinical domain,\u201d","volume-title":"Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing","author":"Romanov","year":"2018"},{"key":"B103","article-title":"Process knowledge-infused learning for suicidality assessment on social media","author":"Roy","year":"","journal-title":"arXiv"},{"key":"B104","volume-title":"Alleviate ChatBot","author":"Roy","year":"2023"},{"key":"B105","doi-asserted-by":"publisher","first-page":"1056728","DOI":"10.3389\/fdata.2022.1056728","article-title":"Proknow: Process knowledge for safety constrained and explainable question generation for mental health diagnostic assistance","volume":"5","author":"Roy","year":"","journal-title":"Front. Big Data"},{"key":"B106","doi-asserted-by":"publisher","first-page":"206","DOI":"10.1038\/s42256-019-0048-x","article-title":"Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead","volume":"1","author":"Rudin","year":"2019","journal-title":"Nat. Mach. Intelli"},{"key":"B107","doi-asserted-by":"publisher","first-page":"887","DOI":"10.3390\/healthcare11060887","article-title":"ChatGPT utility in healthcare education, research, and practice: systematic review on the promising perspectives and valid concerns","volume":"11","author":"Sallam","year":"2023","journal-title":"Healthcare"},{"key":"B108","unstructured":"2020 National Survey of Drug Use and Health (NSDUH) Releases2020"},{"key":"B109","doi-asserted-by":"publisher","first-page":"102848","DOI":"10.1016\/j.ijhcs.2022.102848","article-title":"Can we trust a chatbot like a physician? A qualitative study on understanding the emergence of trust toward diagnostic chatbots","volume":"165","author":"Seitz","year":"2022","journal-title":"Int. J. Hum. Comput. Stud"},{"key":"B110","volume-title":"Towards Facilitating Empathic Conversations in Online Mental Health Support: A Reinforcement Learning Approach","author":"Sharma","year":"2021"},{"key":"B111","doi-asserted-by":"publisher","first-page":"46","DOI":"10.1038\/s42256-022-00593-2","article-title":"Human-ai collaboration enables more empathic conversations in text-based peer-to-peer mental health support","volume":"5","author":"Sharma","year":"2023","journal-title":"Nat. Mach. Intellig"},{"key":"B112","doi-asserted-by":"publisher","first-page":"19","DOI":"10.1109\/MIC.2021.3101919","article-title":"Knowledge-intensive language understanding for explainable ai","volume":"25","author":"Sheth","year":"2021","journal-title":"IEEE Internet Computing"},{"key":"B113","doi-asserted-by":"publisher","first-page":"76","DOI":"10.1109\/MIC.2022.3182349","article-title":"Process knowledge-infused ai: Toward user-level explainability, interpretability, and safety","volume":"26","author":"Sheth","year":"2022","journal-title":"IEEE Inter. Comput"},{"key":"B114","doi-asserted-by":"publisher","first-page":"24","DOI":"10.1109\/MIS.2019.2905748","article-title":"Extending patient-chatbot experience with internet-of-things and background knowledge: case studies with healthcare applications","volume":"34","author":"Sheth","year":"2019","journal-title":"IEEE Intell. Syst"},{"key":"B115","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2005.05716","article-title":"Attviz: Online exploration of self-attention for transparent neural language modeling","author":"\u0160krlj","year":"2020","journal-title":"arXiv"},{"key":"B116","unstructured":"SohailS. H. AI Mental Health Chatbot Diagnoses Disorders with 93% Accuracy2023"},{"key":"B117","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v31i1.11164","article-title":"\u201cConceptnet 5.5: an open multilingual graph of general knowledge,\u201d","author":"Speer","year":"2017","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"B118","doi-asserted-by":"publisher","first-page":"100308","DOI":"10.1016\/j.patter.2021.100308","article-title":"Did chatbots miss their \u201capollo moment\u201d? potential, gaps, and lessons from using collaboration assistants during covid-19","volume":"2","author":"Srivastava","year":"2021","journal-title":"Patterns"},{"key":"B119","doi-asserted-by":"publisher","first-page":"85","DOI":"10.18653\/v1\/2022.naacl-main.6","article-title":"\u201cSemantic diversity in dialogue with natural language inference,\u201d","author":"Stasaski","year":"2022","journal-title":"Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies"},{"key":"B120","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2020.acl-main.634","article-title":"\u201cDiversifying dialogue generation with non-conversational text,\u201d","volume-title":"58th Annual Meeting of the Association for Computational Linguistics","author":"Su","year":"2020"},{"key":"B121","first-page":"3319","article-title":"\u201cAxiomatic attribution for deep networks,\u201d","volume-title":"International Conference on Machine Learning","author":"Sundararajan","year":"2017"},{"key":"B122","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3453175","article-title":"Can chatbots help support a person's mental health? Perceptions and views from mental healthcare professionals and experts","volume":"2","author":"Sweeney","year":"2021","journal-title":"ACM Trans. Comp. Healthcare"},{"key":"B123","doi-asserted-by":"publisher","first-page":"15","DOI":"10.1186\/s40561-023-00237-x","article-title":"What if the devil is my guardian angel: ChatGPT as a case study of using chatbots in education","volume":"10","author":"Tlili","year":"2023","journal-title":"Smart Learn. Environm"},{"key":"B124","doi-asserted-by":"publisher","first-page":"255","DOI":"10.3390\/a15080255","article-title":"Designing reinforcement learning algorithms for digital interventions: pre-implementation guidelines","volume":"15","author":"Trella","year":"2022","journal-title":"Algorithms"},{"key":"B125","doi-asserted-by":"publisher","first-page":"480","DOI":"10.1016\/j.future.2021.05.032","article-title":"An emotion and cognitive based analysis of mental health disorders from social media data","volume":"124","author":"Uban","year":"2021","journal-title":"Future Generat. Computer Syst"},{"key":"B126","author":"Varshney","year":"2021","journal-title":"Trustworthy Machine Learning"},{"key":"B127","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1145\/2629489","article-title":"Wikidata: a free collaborative knowledgebase","volume":"57","author":"Vrande\u010di\u0107","year":"2014","journal-title":"Commun. ACM"},{"key":"B128","article-title":"\u201cPARADISE: a framework for evaluating spoken dialogue agents,\u201d","volume-title":"35th Annual Meeting of the Association for Computational Linguistics and 8th Conference of the European Chapter of the Association for Computational Linguistics","author":"Walker","year":"1997"},{"key":"B129","doi-asserted-by":"publisher","first-page":"2724","DOI":"10.1109\/TKDE.2017.2754499","article-title":"Knowledge graph embedding: a survey of approaches and applications","volume":"29","author":"Wang","year":"2017","journal-title":"IEEE Trans. Knowl. Data Eng"},{"key":"B130","volume-title":"Sensemaking in Organizations","author":"Weick","year":"1995"},{"key":"B131","doi-asserted-by":"crossref","DOI":"10.18653\/v1\/2021.findings-emnlp.210","article-title":"\u201cChallenges in detoxifying language models,\u201d","volume-title":"Findings of the Association for Computational Linguistics: EMNLP 2021","author":"Welbl","year":"2021"},{"key":"B132","first-page":"3315","article-title":"\u201cCurating a large-scale motivational interviewing dataset using peer support forums,\u201d","volume-title":"Proceedings of the 29th International Conference on Computational Linguistics","author":"Welivita","year":""},{"key":"B133","doi-asserted-by":"publisher","first-page":"11459","DOI":"10.1609\/aaai.v36i10.21398","article-title":"\u201cHeal: A knowledge graph for distress management conversations","volume":"36","author":"Welivita","year":"","journal-title":"Proc. AAAI Conf. Artificial Intell"},{"key":"B134","doi-asserted-by":"publisher","first-page":"643","DOI":"10.1177\/070674371105601102","article-title":"Extending motivational interviewing to the treatment of major mental health problems: current directions and evidence","volume":"56","author":"Westra","year":"2011","journal-title":"Canadian J. Psychiat"},{"key":"B135","doi-asserted-by":"publisher","first-page":"54","DOI":"10.1145\/3144592.3144598","article-title":"Why we should have seen that coming: comments on microsoft's tay \u201cexperiment,\u201d and wider implications","volume":"47","author":"Wolf","year":"2017","journal-title":"Acm Sigcas Comp. Soc"},{"key":"B136","first-page":"497","article-title":"\u201cTowards detecting need for empathetic response in motivational interviewing,\u201d","author":"Wu","year":"2020","journal-title":"Companion Publication of the"},{"key":"B137","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2010.07079","article-title":"Recipes for safety in open-domain chatbots","author":"Xu","year":"2020","journal-title":"arXiv"},{"key":"B138","doi-asserted-by":"crossref","DOI":"10.1145\/3110025.3123028","volume-title":"Semi-Supervised Approach to Monitoring Clinical Depressive Symptoms in Social Media","author":"Yazdavar","year":"2017"},{"key":"B139","author":"Zhang","year":"2019","journal-title":"Conversation generation with concept"},{"key":"B140","doi-asserted-by":"publisher","first-page":"46","DOI":"10.1038\/s41746-022-00589-7","article-title":"Natural language processing applied to mental illness detection: a narrative review","volume":"5","author":"Zhang","year":"2022","journal-title":"NPJ Digital Med"},{"key":"B141","doi-asserted-by":"publisher","first-page":"407","DOI":"10.1007\/s11920-022-01350-3","article-title":"Adapting and implementing apps for mental healthcare","volume":"24","author":"Zielasek","year":"2022","journal-title":"Curr. Psychiatry Rep"},{"key":"B142","doi-asserted-by":"crossref","first-page":"30","DOI":"10.18653\/v1\/2022.clpsych-1.3","article-title":"\u201cExplaining models of mental health via clinically grounded auxiliary tasks,\u201d","author":"Zirikly","year":"2022","journal-title":"Proceedings of the Eighth Workshop on Computational Linguistics and Clinical Psychology"}],"container-title":["Frontiers in Artificial Intelligence"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2023.1229805\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,12,23]],"date-time":"2023-12-23T19:54:22Z","timestamp":1703361262000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/frai.2023.1229805\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,12]]},"references-count":142,"alternative-id":["10.3389\/frai.2023.1229805"],"URL":"https:\/\/doi.org\/10.3389\/frai.2023.1229805","relation":{},"ISSN":["2624-8212"],"issn-type":[{"value":"2624-8212","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,12]]},"article-number":"1229805"}}