{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T20:57:35Z","timestamp":1790888255905,"version":"4.1.0"},"reference-count":121,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Res Protoc"],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec sec-type=\"background\">\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Chatbots have the potential to reduce barriers to preexposure prophylaxis (PrEP), including lack of awareness, misconceptions, and stigma, by providing anonymous and continuous support. However, in the context of PrEP, chatbots are still nascent; they lack personalized informational expertise, peer experiential expertise, and human-like emotional support to facilitate future PrEP uptake. These personalized and relatable forms of support are crucial for increasing engagement, influencing health decisions, and fostering resilience and well-being.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"objective\">\n                    <jats:title>Objective<\/jats:title>\n                    <jats:p>In this paper, we describe the iterative development and evaluation plans of a retrieval-augmented generation (RAG) chatbot for providing personalized information, peer experiential expertise, and human-like emotional support to PrEP candidates.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"methods\">\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We used an iterative design process consisting of 2 phases: prototype conceptualization and iterative chatbot development. In the conceptualization phase, we identified real-world PrEP needs and designed a functional dialogue flow diagram for PrEP support. Chatbot development included developing 2 components: a query preprocessor and a RAG module. The preprocessor uses the Segment Any Text (SAT) tool (developed by Markus Frohmann, Igor Sterner, Ivan Vuli\u0107, Benjamin Minixhofer, and Markus Schedl) for query segmentation and a Gemma 2 fine-tuned support classifier to identify informational, emotional, and contextual data from real-world queries. To implement the RAG module, we used Sentence-Bidirectional Encoder Representations from Transformers (SBERT) embeddings with cosine similarity, and performed topic matching to identify topically relevant documents based on the query topic to support document retrieval. Extensive prompt engineering was used to guide the large language model (LLM), Gemini-2.0-flash, in generating tailored responses. We conducted 10 rounds of internal evaluations to assess and improve the chatbot responses based on 10 criteria: clarity, accuracy, actionability, relevancy, information detail, tailored information, comprehensiveness, language suitability, tone, and empathy. Finally, we conducted a blinded comparative study and used a linear mixed model to validate the chatbot against a general LLM and real-world user responses.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"results\">\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>We developed a RAG chatbot and iteratively refined it based on the internal evaluation feedback. Prompt engineering is essential in guiding the LLM to generate responses tailored to information, experiential, and emotional user needs. We found that prompt effectiveness varied with task complexity; this was likely due to LLM sensitivity to the structure of prompts and to linguistic variability. Prompt decomposition and segmenting prompt instructions helped improve comprehensiveness and relevancy for complex and long queries. The linear mixed model analysis revealed that while both the general LLM and RAG were preferred over user responses, the general LLM maintained a superior edge over the RAG chatbot across most criteria, with the exceptions of empathy and tone.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec sec-type=\"conclusions\">\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>Our RAG chatbot leverages social media data to provide personalized information, peer experiences, and human-like emotional support; these elements are essential for addressing PrEP misconceptions and promoting self-efficacy. Further analysis incorporating expert and user feedback will be conducted to help validate and improve the chatbot\u2019s potential.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.2196\/79966","type":"journal-article","created":{"date-parts":[[2026,6,3]],"date-time":"2026-06-03T07:00:10Z","timestamp":1780470010000},"page":"e79966-e79966","source":"Crossref","is-referenced-by-count":0,"title":["A Conversational Agent for Providing Personalized Preexposure Prophylaxis (PrEP) Support: Protocol for Chatbot Implementation and Evaluation"],"prefix":"10.2196","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-3932-7727","authenticated-orcid":false,"given":"Fatima","family":"Sayed","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2419-0131","authenticated-orcid":false,"given":"Albert","family":"Park","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7728-0587","authenticated-orcid":false,"given":"Patrick Sean","family":"Sullivan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9576-0293","authenticated-orcid":false,"given":"Yaorong","family":"Ge","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,10,1]]},"reference":[{"issue":"6","key":"R1","doi-asserted-by":"publisher","first-page":"274","DOI":"10.14745\/ccdr.v49i06a04","article-title":"Increased PrEP uptake and PrEP-RN coincide with decreased HIV diagnoses in men who have sex with men in Ottawa, Canada","volume":"49","author":"Kroch","journal-title":"Can Commun Dis Rep"},{"issue":"7","key":"R2","doi-asserted-by":"publisher","first-page":"e528","DOI":"10.1016\/S2468-2667(21)00112-2","article-title":"HIV pre-exposure prophylaxis: scaling up for impact now and in the future","volume":"6","author":"Bavinton","journal-title":"Lancet Public Health"},{"issue":"4","key":"R3","doi-asserted-by":"publisher","first-page":"444","DOI":"10.1097\/QAI.0000000000000993","article-title":"PrEP adherence patterns strongly affect individual HIV risk and observed efficacy in randomized clinical trials","volume":"72","author":"Dimitrov","journal-title":"J Acquir Immune Defic Syndr"},{"key":"R4","unstructured":"EHE overview. HIV.gov. URL: https:\/\/www.hiv.gov\/federal-response\/ending-the-hiv-epidemic\/overview [Accessed 12-04-2025]"},{"key":"R5","doi-asserted-by":"publisher","DOI":"10.1016\/j.lana.2024.100738","article-title":"Equity of PrEP uptake by race, ethnicity, sex and region in the United States in the first decade of PrEP: a population-based analysis","volume":"33","author":"Sullivan","journal-title":"Lancet Reg Health Am"},{"issue":"5","key":"R6","doi-asserted-by":"publisher","first-page":"1778","DOI":"10.1007\/s12325-020-01295-0","article-title":"Barriers to the wider use of pre-exposure prophylaxis in the United States: a narrative review","volume":"37","author":"Mayer","journal-title":"Adv Ther"},{"issue":"1","key":"R7","doi-asserted-by":"publisher","DOI":"10.1186\/s12889-023-15260-3","article-title":"Barriers and facilitators to oral PrEP uptake among high-risk men after HIV testing at workplaces in Uganda: a qualitative study","volume":"23","author":"Nabunya","journal-title":"BMC Public Health"},{"key":"R8","doi-asserted-by":"publisher","DOI":"10.3389\/frph.2023.1084657","article-title":"\u201cPrEP protects us\u201d: behavioural, normative, and control beliefs influencing pre-exposure prophylaxis uptake among pregnant and breastfeeding women in Zambia","volume":"5","author":"Hamoonga","journal-title":"Front Reprod Health"},{"key":"R9","doi-asserted-by":"publisher","DOI":"10.1016\/j.pmedr.2022.102062","article-title":"Barriers to PrEP uptake among Black female adolescents and emerging adults","volume":"31","author":"Crooks","journal-title":"Prev Med Rep"},{"issue":"2","key":"R10","doi-asserted-by":"publisher","first-page":"186","DOI":"10.1037\/prj0000109","article-title":"Peer education as a strategy for reducing internalized stigma among depressed older adults","volume":"38","author":"Conner","journal-title":"Psychiatr Rehabil J"},{"issue":"9","key":"R11","doi-asserted-by":"publisher","first-page":"2995","DOI":"10.3390\/ijerph17092995","article-title":"How could peers in online health community help improve health behavior","volume":"17","author":"Li","journal-title":"Int J Environ Res Public Health"},{"issue":"4","key":"R12","doi-asserted-by":"publisher","first-page":"227","DOI":"10.1207\/S15327965PLI1104_01","article-title":"The \u201cWhat\u201d and \u201cWhy\u201d of goal pursuits: human needs and the self-determination of behavior","volume":"11","author":"Deci","journal-title":"Psychol Inq"},{"issue":"3","key":"R13","doi-asserted-by":"publisher","first-page":"337","DOI":"10.1037\/sah0000148","article-title":"The effects of peer support on empowerment, self-efficacy, and internalized stigma: a narrative synthesis and meta-analysis","volume":"4","author":"Burke","journal-title":"Stigma Health"},{"issue":"9","key":"R14","doi-asserted-by":"publisher","first-page":"769","DOI":"10.1056\/NEJMp2216100","article-title":"Investing in national HIV PrEP preparedness","volume":"388","author":"Johnson","journal-title":"N Engl J Med"},{"issue":"3","key":"R15","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0283764","article-title":"Ending the HIV epidemic PrEP equity recommendations from a rapid ethnographic assessment of multilevel PrEP use determinants among young Black gay and bisexual men in Atlanta, GA","volume":"18","author":"Hill","journal-title":"PLoS One"},{"issue":"6","key":"R16","doi-asserted-by":"publisher","first-page":"522","DOI":"10.1071\/SH18103","article-title":"Getting pre-exposure prophylaxis (PrEP) to the people: opportunities, challenges and emerging models of PrEP implementation","volume":"15","author":"Sullivan","journal-title":"Sex Health"},{"issue":"3","key":"R17","doi-asserted-by":"publisher","first-page":"130","DOI":"10.7812\/TPP\/23.134","article-title":"The impact of digital health solutions on bridging the health care gap in rural areas: a scoping review","volume":"28","author":"Maita","journal-title":"Perm J"},{"issue":"1","key":"R18","doi-asserted-by":"publisher","DOI":"10.2196\/10659","article-title":"Developing a mobile app (LYNX) to support linkage to HIV\/sexually transmitted infection testing and pre-exposure prophylaxis for young men who have sex with men: protocol for a randomized controlled trial","volume":"8","author":"Liu","journal-title":"JMIR Res Protoc"},{"issue":"1","key":"R19","doi-asserted-by":"publisher","DOI":"10.2196\/42983","article-title":"Increasing participation in a telePrEP program for sexual and gender minority adolescents and young adults in louisiana: protocol for an SMS text messaging-based chatbot","volume":"12","author":"Braddock","journal-title":"JMIR Res Protoc"},{"issue":"12","key":"R20","doi-asserted-by":"publisher","DOI":"10.2196\/39816","article-title":"The feasibility and acceptability of an mHealth conversational agent designed to support HIV self-testing in South Africa: cross-sectional study","volume":"24","author":"Ntinga","journal-title":"J Med Internet Res"},{"key":"R21","doi-asserted-by":"publisher","DOI":"10.2196\/48447","article-title":"Evaluating an innovative HIV self-testing service with web-based, real-time counseling provided by an artificial intelligence chatbot (HIVST-chatbot) in increasing HIV self-testing use among Chinese men who have sex with men: protocol for a noninferiority randomized controlled trial","volume":"12","author":"Chen","journal-title":"JMIR Res Protoc"},{"issue":"1","key":"R22","doi-asserted-by":"publisher","DOI":"10.2196\/54073","article-title":"Adapting mHealth interventions (PrEPmate and DOT diary) to support PrEP retention in care and adherence among English and Spanish-speaking men who have sex with men and transgender women in the United States: formative work and pilot randomized trial","volume":"8","author":"Liu","journal-title":"JMIR Form Res"},{"key":"R23","doi-asserted-by":"publisher","DOI":"10.2196\/41881","article-title":"A transgender chatbot (Amanda Selfie) to create pre-exposure prophylaxis demand among adolescents in Brazil: assessment of acceptability, functionality, usability, and results","volume":"25","author":"Massa","journal-title":"J Med Internet Res"},{"issue":"1","key":"R24","doi-asserted-by":"publisher","DOI":"10.2196\/59975","article-title":"Ameliorating racial disparities in HIV prevention via a nurse-led, AI-enhanced program for pre-exposure prophylaxis utilization among black cisgender women: protocol for a mixed methods study","volume":"13","author":"Zhang","journal-title":"JMIR Res Protoc"},{"key":"R25","volume":"10","author":"Muessig","journal-title":"Digit Cult Educ"},{"issue":"9","key":"R26","doi-asserted-by":"publisher","DOI":"10.2196\/38354","article-title":"Tough talks virtual simulation HIV disclosure intervention for young men who have sex with men: development and usability testing","volume":"6","author":"Hightow-Weidman","journal-title":"JMIR Form Res"},{"key":"R27","doi-asserted-by":"publisher","DOI":"10.2196\/55559","article-title":"Development and pilot-testing of an optimized conversational agent or \u201cChatbot\u201d for peruvian adolescents living with HIV to facilitate mental health screening, education, self-help, and linkage to care: protocol for a mixed methods, community-engaged study","volume":"13","author":"Galea","journal-title":"JMIR Res Protoc"},{"issue":"4","key":"R28","doi-asserted-by":"publisher","DOI":"10.2196\/27850","article-title":"Chatbot for health care and oncology applications using artificial intelligence and machine learning: systematic review","volume":"7","author":"Xu","journal-title":"JMIR Cancer"},{"issue":"1","key":"R29","doi-asserted-by":"publisher","first-page":"012041","DOI":"10.1088\/1742-6596\/1469\/1\/012041","article-title":"Mobile based chatbot application for HIV\/AIDS counseling using artificial intelligence markup language approach","volume":"1469","author":"Ardiana","journal-title":"J Phys: Conf Ser"},{"key":"R30","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2023.126902","article-title":"An intelligent conversational agent for educating the general public about HIV","volume":"563","author":"Moreno","journal-title":"Neurocomputing"},{"issue":"4","key":"R31","doi-asserted-by":"publisher","DOI":"10.2196\/15806","article-title":"A smartphone-based health care chatbot to promote self-management of chronic pain (SELMA): pilot randomized controlled trial","volume":"8","author":"Hauser-Ulrich","journal-title":"JMIR Mhealth Uhealth"},{"issue":"159","key":"R32","doi-asserted-by":"publisher","DOI":"10.3791\/60794","article-title":"Implementation of a real-time psychosis risk detection and alerting system based on electronic health records using CogStack","volume":"PMID","author":"Wang","journal-title":"J Vis Exp"},{"issue":"4","key":"R33","doi-asserted-by":"publisher","DOI":"10.2196\/mental.9782","article-title":"Using psychological artificial intelligence (Tess) to relieve symptoms of depression and anxiety: randomized controlled trial","volume":"5","author":"Fulmer","journal-title":"JMIR Ment Health"},{"key":"R34","doi-asserted-by":"publisher","DOI":"10.1016\/j.invent.2022.100495","article-title":"Using AI chatbots to provide self-help depression interventions for university students: a randomized trial of effectiveness","volume":"27","author":"Liu","journal-title":"Internet Interv"},{"key":"R35","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijmedinf.2021.104440","article-title":"Mobile app-based chatbot to deliver cognitive behavioral therapy and psychoeducation for adults with attention deficit: a development and feasibility\/usability study","volume":"150","author":"Jang","journal-title":"Int J Med Inform"},{"key":"R36","doi-asserted-by":"publisher","DOI":"10.1016\/j.ajo.2020.09.008","article-title":"Text parsing-based identification of patients with poor glaucoma medication adherence in the electronic health record","volume":"222","author":"Hamid","journal-title":"Am J Ophthalmol"},{"key":"R37","doi-asserted-by":"crossref","unstructured":"De Nieva JO Joaquin JA Tan CB Marc Te RK Ong E . Investigating students\u2019 use of a mental health chatbot to alleviate academic stress. Presented at: CHIuXiD \u201920: 6th International ACM In-Cooperation HCI and UX Conference; Oct 21-23, 2020; Jakarta & Bandung Indonesia. URL: https:\/\/dl.acm.org\/doi\/proceedings\/10.1145\/3431656 [Accessed 12-08-2026]","DOI":"10.1145\/3431656.3431657"},{"issue":"5","key":"R38","doi-asserted-by":"publisher","first-page":"507","DOI":"10.1136\/jamia.2009.001560","article-title":"Mayo clinical Text Analysis and Knowledge Extraction System (cTAKES): architecture, component evaluation and applications","volume":"17","author":"Savova","journal-title":"J Am Med Inform Assoc"},{"key":"R39","doi-asserted-by":"crossref","unstructured":"Ali MR Razavi SZ Langevin R et al. A virtual conversational agent for teens with autism spectrum disorder: experimental results and design lessons. Presented at: IVA \u201920: 20th ACM International Conference on Intelligent Virtual Agents; Oct 19-23, 2020. URL: https:\/\/dl.acm.org\/doi\/10.1145\/3383652.3423900 [Accessed 19-09-2026] [doi: 10.1145\/3383652.3423900]","DOI":"10.1145\/3383652.3423900"},{"issue":"10","key":"R40","doi-asserted-by":"publisher","DOI":"10.2196\/42055","article-title":"Formative evaluation of the acceptance of HIV prevention artificial intelligence chatbots by men who have sex with men in Malaysia: focus group study","volume":"6","author":"Peng","journal-title":"JMIR Form Res"},{"issue":"5","key":"R41","doi-asserted-by":"publisher","first-page":"385","DOI":"10.1071\/SH21123","article-title":"Barriers and facilitators to engagement with artificial intelligence (AI)-based chatbots for sexual and reproductive health advice: a qualitative analysis","volume":"18","author":"Nadarzynski","journal-title":"Sex Health"},{"key":"R42","doi-asserted-by":"crossref","unstructured":"Luo M Warren CJ Cheng L Abdul-Muhsin HM Banerjee I . Assessing empathy in large language models with real-world physician-patient interactions. Presented at: 2024 IEEE International Conference on Big Data (BigData); Dec 15-18, 2024:6510-6519; Washington, DC, USA. 2024.URL: https:\/\/ieeexplore.ieee.org\/document\/10825307 [Accessed 19-09-2026] [doi: 10.1109\/BigData62323.2024.10825307]","DOI":"10.1109\/BigData62323.2024.10825307"},{"issue":"2","key":"R43","doi-asserted-by":"publisher","first-page":"85","DOI":"10.11621\/pir.2024.0206","article-title":"The emotional intelligence of the GPT-4 large language model","volume":"17","author":"Vzorin","journal-title":"Psychol Russ"},{"issue":"1","key":"R44","doi-asserted-by":"publisher","DOI":"10.2196\/52597","article-title":"Large language models and empath: systematic review","volume":"26","author":"Sorin","journal-title":"J Med Internet Res"},{"issue":"4","key":"R45","doi-asserted-by":"publisher","first-page":"1353","DOI":"10.1007\/s00146-021-01230-z","article-title":"In principle obstacles for empathic AI: why we can\u2019t replace human empathy in healthcare","volume":"37","author":"Montemayor","journal-title":"AI Soc"},{"key":"R46","doi-asserted-by":"publisher","DOI":"10.21037\/jmai-23-136","article-title":"An experimental study of integrating fine-tuned large language models and prompts for enhancing mental health support chatbot system","volume":"7","author":"Yu","journal-title":"J Med Artif Intell"},{"key":"R47","unstructured":"Ye H Liu T Zhang A Hua W Jia W . Cognitive mirage: a review of hallucinations in large language models. arXiv. Preprint posted online on Sep 13, 2023. [doi: 10.48550\/arXiv.2309.06794]"},{"key":"R48","unstructured":"Gilson A Ai X Arunachalam T et al. Enhancing large language models with domain-specific retrieval augment generation: a case study on long-form consumer health question answering in ophthalmology. arXiv. Preprint posted online on Sep 20, 2024. [doi: 10.48550\/arXiv.2409.13902]"},{"issue":"1","key":"R49","doi-asserted-by":"publisher","first-page":"100190","DOI":"10.1016\/j.metrad.2025.100190","article-title":"Prompt engineering for healthcare: methodologies and applications","volume":"4","author":"Wang","journal-title":"Meta-Radiology"},{"issue":"8","key":"R50","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2024.25981","article-title":"Use of generative AI to identify helmet status among patients with micromobility-related injuries from unstructured clinical notes","volume":"7","author":"Burford","journal-title":"JAMA Netw Open"},{"issue":"1","key":"R51","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-023-00970-0","article-title":"Large language models to identify social determinants of health in electronic health records","volume":"7","author":"Guevara","journal-title":"NPJ Digit Med"},{"key":"R52","unstructured":"Yu H Guo P Sano A . Zero-shot ECG diagnosis with large language models and retrieval-augmented generation. Presented at: Proceedings of the 3rd Machine Learning for Health Symposium, PMLR. 650-663. 2023."},{"key":"R53","doi-asserted-by":"crossref","unstructured":"Zeng S Zhang J He P et al. Mitigating the privacy issues in retrieval-augmented generation (RAG) via pure synthetic data. Presented at: 2025 Conference on Empirical Methods in Natural Language Processing; Nov 4-7, 2025; Suzhou, China. URL: https:\/\/aclanthology.org\/2025.emnlp-main [Accessed 12-08-2026] [doi: 10.18653\/v1\/2025.emnlp-main.1247]","DOI":"10.18653\/v1\/2025.emnlp-main.1247"},{"key":"R54","doi-asserted-by":"crossref","unstructured":"Salemi A Mysore S Bendersky M Zamani H . LaMP: when large language models meet personalization. Presented at: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers); Aug 11-16, 2024; Bangkok, Thailand. URL: https:\/\/aclanthology.org\/2024.acl-long.399\/ [Accessed 19-09-2026] [doi: 10.18653\/v1\/2024.acl-long.399]","DOI":"10.18653\/v1\/2024.acl-long.399"},{"issue":"4","key":"R55","doi-asserted-by":"publisher","first-page":"42","DOI":"10.1007\/s11280-024-01276-1","article-title":"When large language models meet personalization: perspectives of challenges and opportunities","volume":"27","author":"Chen","journal-title":"World Wide Web"},{"key":"R56","doi-asserted-by":"crossref","unstructured":"Shi W Zhuang Y Zhu Y Iwinski H Wattenbarger M Wang MD . Retrieval-augmented large language models for adolescent idiopathic scoliosis patients in shared decision-making. Presented at: BCB\u201923: 14th ACM International Conference on Bioinformatics, Computational Biology, and Health Informatics; Sep 3-6, 2023:1-10; Houston, TX. URL: https:\/\/dl.acm.org\/doi\/proceedings\/10.1145\/3584371 [Accessed 19-09-2026] [doi: 10.1145\/3584371.3612956]","DOI":"10.1145\/3584371.3612956"},{"issue":"2","key":"R57","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3729479","article-title":"CataractBot: an LLM-powered expert-in-the-loop chatbot for cataract patients","volume":"9","author":"Ramjee","journal-title":"Proc ACM Interact Mob Wearable Ubiquitous Technol"},{"issue":"1","key":"R58","doi-asserted-by":"publisher","DOI":"10.1056\/AIra2400380","article-title":"RAG in health care: a novel framework for improving communication and decision-making by addressing LLM limitations","volume":"2","author":"Ng","journal-title":"NEJM AI"},{"key":"R59","doi-asserted-by":"publisher","DOI":"10.3389\/fmed.2024.1392555","article-title":"GastroBot: a Chinese gastrointestinal disease chatbot based on the retrieval-augmented generation","volume":"11","author":"Zhou","journal-title":"Front Med (Lausanne)"},{"issue":"6","key":"R60","doi-asserted-by":"publisher","first-page":"523","DOI":"10.1007\/s10865-009-9222-7","article-title":"Emotional support and gender in people living with HIV: effects on psychological well-being","volume":"32","author":"Gordillo","journal-title":"J Behav Med"},{"issue":"5","key":"R61","doi-asserted-by":"publisher","first-page":"2649","DOI":"10.1007\/s40615-023-01729-9","article-title":"Barriers and facilitators to acceptability and uptake of pre-exposure prophylaxis (PrEP) among Black women in the United States: a systematic review","volume":"11","author":"Smit","journal-title":"J Racial and Ethnic Health Disparities"},{"key":"R62","doi-asserted-by":"publisher","DOI":"10.3389\/fpubh.2021.667331","article-title":"Pills, PrEP, and Pals: adherence, stigma, resilience, faith and the need to connect among minority women with HIV\/AIDS in a US HIV epicenter","volume":"9","author":"Duthely","journal-title":"Front Public Health"},{"issue":"3","key":"R63","doi-asserted-by":"publisher","DOI":"10.2196\/jmir.1728","article-title":"Managing the personal side of health: how patient expertise differs from the expertise of clinicians","volume":"13","author":"Hartzler","journal-title":"J Med Internet Res"},{"issue":"3","key":"R64","doi-asserted-by":"publisher","first-page":"321","DOI":"10.1016\/s0020-7489(02)00092-5","article-title":"Peer support within a health care context: a concept analysis","volume":"40","author":"Dennis","journal-title":"Int J Nurs Stud"},{"key":"R65","doi-asserted-by":"crossref","unstructured":"Wang YC Kraut R Levine JM . To stay or leave? the relationship of emotional and informational support to commitment in online health support groups. Presented at: CSCW\u201912: Proceedings of the ACM Conference on Computer Supported Cooperative Work; Feb 11-15, 2012:833-842; Seattle, WA. URL: https:\/\/dl.acm.org\/doi\/10.1145\/2145204.2145329 [Accessed 19-09-2026] [doi: 10.1145\/2145204.2145329]","DOI":"10.1145\/2145204.2145329"},{"issue":"3","key":"R66","doi-asserted-by":"publisher","first-page":"307","DOI":"10.1057\/s41285-018-0081-6","article-title":"Patients\u2019 experiential knowledge and expertise in health care: a hybrid concept analysis","volume":"17","author":"Castro","journal-title":"Soc Theory Health"},{"key":"R67","doi-asserted-by":"crossref","unstructured":"Burleson BR . Emotional support skills. In: Greene JO Burleson BR , editors. Handbook of Communication and Social Interaction Skills Routledge. Lawrence Erlbaum Associates, Inc; 2003. [doi: 10.4324\/9781410607133-22]","DOI":"10.4324\/9781410607133-22"},{"issue":"4","key":"R68","doi-asserted-by":"publisher","first-page":"344","DOI":"10.1511\/2001.28.344","article-title":"The nature of emotions: human emotions have deep evolutionary roots, a fact that may explain their complexity and provide tools for clinical practice","volume":"89","author":"Plutchik","journal-title":"Am Sci"},{"issue":"4","key":"R69","doi-asserted-by":"publisher","first-page":"274","DOI":"10.1177\/036215378201200411","article-title":"The feeling wheel: a tool for expanding awareness of emotions and increasing spontaneity and intimacy","volume":"12","author":"Willcox","journal-title":"Trans Anal J"},{"issue":"2","key":"R70","doi-asserted-by":"publisher","DOI":"10.2196\/ijmr.7000","article-title":"How, when and why people seek health information online: qualitative study in Hong Kong","volume":"6","author":"Chu","journal-title":"Interact J Med Res"},{"key":"R71","doi-asserted-by":"crossref","unstructured":"Aliannejadi M Chakraborty M R\u00edssola EA Crestani F . Harnessing evolution of multi-turn conversations for effective answer retrieval. Presented at: CHIR \u201920: Conference on Human Information Interaction and Retrieval; Aug 14-18, 2020:33-42; Vancouver, BC, Canada. URL: https:\/\/dl.acm.org\/doi\/proceedings\/10.1145\/3343413 [Accessed 12-08-2026] [doi: 10.1145\/3343413.3377968]","DOI":"10.1145\/3343413.3377968"},{"issue":"2","key":"R72","doi-asserted-by":"publisher","first-page":"151","DOI":"10.1007\/s00127-007-0277-x","article-title":"Health-related quality of life and health behaviors by social and emotional support","volume":"43","author":"Strine","journal-title":"Soc Psychiat Epidemiol"},{"key":"R73","doi-asserted-by":"publisher","DOI":"10.2196\/58041","article-title":"Enhancement of the performance of large language models in diabetes education through retrieval-augmented generation: comparative study","volume":"26","author":"Wang","journal-title":"J Med Internet Res"},{"issue":"2","key":"R74","doi-asserted-by":"publisher","first-page":"154","DOI":"10.1177\/009365092019002002","article-title":"Controllability of stressful events and satisfaction with spouse support behaviors","volume":"19","author":"Cutrona","journal-title":"Communic Res"},{"key":"R75","unstructured":"Suhr JA Cutrona CE Krebs KK Jensen SL . The social support behavior code (SSBC). In: Couple Observational Coding Systems. Routedge; 2004. URL: https:\/\/www.taylorfrancis.com\/chapters\/edit\/10.4324\/9781410610843-24\/social-support-behavior-code-ssbc-julie-suhr-carolyn-cutrona-krista-krebs-sandra-jensen [Accessed 10-08-2026]"},{"issue":"10","key":"R76","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0258542","article-title":"Adolescent girls and young women\u2019s PrEP-user journey during an implementation science study in South Africa and Kenya","volume":"16","author":"Rousseau","journal-title":"PLoS One"},{"key":"R77","doi-asserted-by":"crossref","unstructured":"Frohmann M Sterner I Vuli\u0107 I Minixhofer B Schedl M . Segment any text: a universal approach for robust, efficient and adaptable sentence segmentation. Presented at: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing; Nov 12-16, 2024; Miami, Florida, USA. URL: https:\/\/aclanthology.org\/2024.emnlp-main.665\/ [Accessed 19-06-2026] [doi: 10.18653\/v1\/2024.emnlp-main.665]","DOI":"10.18653\/v1\/2024.emnlp-main.665"},{"key":"R78","unstructured":"Team G Riviere M Pathak S et al. Gemma 2: improving open language models at a practical size. arXiv. Preprint posted online on Oct 2, 2024. [doi: 10.48550\/arXiv.2408.00118]"},{"key":"R79","unstructured":"Lialin V Deshpande V Rumshisky A . Scaling down to scale up: a guide to parameter-efficient fine-tuning. arXiv. Preprint posted online on Mar 28, 2023. [doi: 10.48550\/arXiv.2303.15647]"},{"key":"R80","doi-asserted-by":"crossref","unstructured":"Malladi S Gao T Nichani E et al. Fine-tuning language models with just forward passes. Presented at: The 37th Annual Conference on Neural Information Processing Systems (NeurIPS 2023); Dec 10-16, 2023; New Orleans, Louisiana, USA. URL: https:\/\/proceedings.neurips.cc\/paper_files\/paper\/2023\/file\/a627810151be4d13f907ac898ff7e948-Paper-Conference.pdf? [Accessed 19-09-2026] [doi: 10.52202\/075280-2308]","DOI":"10.52202\/075280-2308"},{"key":"R81","unstructured":"Maatouk A Ampudia KC Ying R Tassiulas L . Tele-LLMs: a series of specialized large language models for telecommunications. arXiv. Preprint posted online on Sep 9, 2024. [Accessed 21-09-2026] [doi: 10.48550\/arXiv.2409.05314]"},{"key":"R82","doi-asserted-by":"crossref","unstructured":"Reimers N Gurevych I . Sentence-BERT: sentence embeddings using siamese BERT-networks. Presented at: 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP); Nov 3-7, 2019; Hong Kong, China. URL: https:\/\/www.aclweb.org\/anthology\/D19-1 [Accessed 12-08-2026] [doi: 10.18653\/v1\/D19-1410]","DOI":"10.18653\/v1\/D19-1410"},{"key":"R83","doi-asserted-by":"crossref","unstructured":"Mahajan Y Bansal N Blanco E Karmaker S . ALIGN-SIM: a task-free test bed for evaluating and interpreting sentence embeddings through semantic similarity alignment. Presented at: Findings of the Association for Computational Linguistics; Nov 12-16, 2024:7393-7428; Miami, Florida, USA. URL: https:\/\/aclanthology.org\/2024.findings-emnlp.436\/ [Accessed 19-09-2026] [doi: 10.18653\/v1\/2024.findings-emnlp.436]","DOI":"10.18653\/v1\/2024.findings-emnlp.436"},{"issue":"7","key":"R84","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3462476","article-title":"Semantic information retrieval on medical texts","volume":"54","author":"Tamine","journal-title":"ACM Comput Surv"},{"key":"R85","doi-asserted-by":"publisher","DOI":"10.1162\/TACL.a.19","article-title":"mt RAG: a multi-turn conversational benchmark for evaluating retrieval-augmented generation systems","volume":"13","author":"Katsis","journal-title":"Trans Assoc Comput Linguist"},{"key":"R86","doi-asserted-by":"crossref","unstructured":"Radlinski F Craswell N . Comparing the sensitivity of information retrieval metrics. Presented at: The 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR \u201910); Jul 19-23, 2010:667-674; Geneva, Switzerland. [doi: 10.1145\/1835449.1835560]","DOI":"10.1145\/1835449.1835560"},{"issue":"10","key":"R87","doi-asserted-by":"publisher","first-page":"2816","DOI":"10.1007\/s10461-019-02466-z","article-title":"Emotional communication in HIV care: an observational study of patients\u2019 expressed emotions and clinician response","volume":"23","author":"Park","journal-title":"AIDS Behav"},{"key":"R88","unstructured":"Vamsi\/t5_paraphrase_paws. Hugging Face. URL: https:\/\/huggingface.co\/Vamsi\/T5_Paraphrase_Paws [Accessed 09-03-2025]"},{"issue":"1","key":"R89","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01074-z","article-title":"Foundation metrics for evaluating effectiveness of healthcare conversations powered by generative AI","volume":"7","author":"Abbasian","journal-title":"NPJ Digit Med"},{"key":"R90","unstructured":"Huang J Lam MH Li EJ et al. Emotionally numb or empathetic? Evaluating how LLMs feel using EmotionBench. arXiv. Preprint posted online on Nov 16, 2023. [Accessed 21-09-2026] [doi: 10.48550\/arXiv.2308.03656]"},{"key":"R91","doi-asserted-by":"crossref","unstructured":"Wang Y Wang M Manzoor MA et al. Factuality of large language models: a survey. Presented at: 2024 Conference on Empirical Methods in Natural Language Processing; Nov 12-16, 2024; Miami, FL. URL: https:\/\/aclanthology.org\/2024.emnlp-main [Accessed 12-08-2026] [doi: 10.18653\/v1\/2024.emnlp-main.1088]","DOI":"10.18653\/v1\/2024.emnlp-main.1088"},{"key":"R92","doi-asserted-by":"publisher","DOI":"10.1101\/2023.04.18.23288752","article-title":"Faithful AI in medicine: a systematic review with large language models and beyond","author":"Xie","journal-title":"medRxiv"},{"key":"R93","unstructured":"Hewett TT . The role of iterative evaluation in designing systems for usability. In: Proceedings of the Second Conference of the British Computer Society, Human Computer Interaction Specialist Group on People and Computers: Designing for Usability. Cambridge University Press; 1986:196-214. [doi: 10.5555\/17324.24085]"},{"issue":"3","key":"R94","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3641289","article-title":"A survey on evaluation of large language models","volume":"15","author":"Chang","journal-title":"ACM Trans Intell Syst Technol"},{"key":"R95","doi-asserted-by":"crossref","unstructured":"van der Lee C Gatt A van Miltenburg E Wubben S Krahmer E . Best practices for the human evaluation of automatically generated text. Presented at: 12th International Conference on Natural Language Generation; Oct 29 to Nov 1, 2019; Tokyo, Japan. URL: https:\/\/aclanthology.org\/W19-86 [Accessed 12-08-2026] [doi: 10.18653\/v1\/W19-8643]","DOI":"10.18653\/v1\/W19-8643"},{"issue":"10","key":"R96","doi-asserted-by":"publisher","first-page":"1437","DOI":"10.1001\/jamaoncol.2023.2947","article-title":"Assessment of artificial intelligence chatbot responses to top searched queries about cancer","volume":"9","author":"Pan","journal-title":"JAMA Oncol"},{"issue":"3","key":"R97","doi-asserted-by":"publisher","first-page":"198","DOI":"10.3390\/ime2030019","article-title":"Prompt engineering in medical education","volume":"2","author":"Heston","journal-title":"Int Med Educ"},{"issue":"9","key":"R98","doi-asserted-by":"publisher","first-page":"830","DOI":"10.1056\/NEJMoa1600693","article-title":"Antiretroviral therapy for the prevention of HIV-1 transmission","volume":"375","author":"Cohen","journal-title":"N Engl J Med"},{"issue":"183","key":"R99","doi-asserted-by":"publisher","first-page":"589","DOI":"10.1001\/jamainternmed.2023.1838","article-title":"Responses to patient questions posted to a public social media forum","volume":"71","author":"Ayers","journal-title":"JAMA Intern Med"},{"key":"R100","unstructured":"Zhao S Huang Y Song J Wang Z Wan C Ma L . Understanding the fundamental design decisions of retrieval-augmented generation systems. arXiv. Preprint posted online on Nov 29, 2024. [doi: 10.48550\/arXiv.2411.19463]"},{"issue":"2","key":"R101","doi-asserted-by":"publisher","first-page":"155","DOI":"10.1016\/j.jcm.2016.02.012","article-title":"A guideline of selecting and reporting intraclass correlation coefficients for reliability research","volume":"15","author":"Koo","journal-title":"J Chiropr Med"},{"key":"R102","unstructured":"Lin CY . ROUGE: a package for automatic evaluation of summaries. Presented at: Text Summarization Branches Out (A Post-Conference Workshop of ACL 2004); Jul 25, 2004:74-81; Barcelona, Spain. URL: https:\/\/aclanthology.org\/W04-1013\/ [Accessed 09-03-2025]"},{"key":"R103","doi-asserted-by":"crossref","unstructured":"Meincke L Mollick ER Mollick L Shapiro D . Prompting science report 1: prompt engineering is complicated and contingent. Social Science Research Network; 2025. URL: https:\/\/papers.ssrn.com\/abstract=5165270 [Accessed 10-08-2026] [doi: 10.2139\/ssrn.5165270]","DOI":"10.2139\/ssrn.5165270"},{"key":"R104","doi-asserted-by":"crossref","unstructured":"Leidinger A van Rooij R Shutova E . The language of prompting: what linguistic properties make a prompt successful? Presented at: Findings of the Association for Computational Linguistics: EMNLP 2023; Dec 6-10, 2023:9210-9232; Singapore. URL: https:\/\/aclanthology.org\/2023.findings-emnlp.618.pdf [Accessed 20-09-2026] [doi: 10.18653\/v1\/2023.findings-emnlp.618]","DOI":"10.18653\/v1\/2023.findings-emnlp.618"},{"key":"R105","unstructured":"Lamichhane B . Evaluation of chatgpt for NLP-based mental health applications. arXiv. Preprint posted online on Mar 28, 2023. [doi: 10.48550\/arXiv.2303.15727]"},{"key":"R106","unstructured":"Tay W Joshi A Zhang X Karimi S Wan S . Red-faced ROUGH: examining the suitability of ROUGE for opinion summary evaluation. In: Mistica M Piccardi M MacKinlay A , editors. Presented at: The 17th Annual Workshop of the Australasian Language Technology Association (ALTA 2019); Dec 4-6, 2019; Sydney, Australia. URL: https:\/\/aclanthology.org\/U19-1008\/ [Accessed 20-09-2026]"},{"key":"R107","doi-asserted-by":"crossref","unstructured":"Wallace E Wang Y Li S Singh S Gardner M . Do NLP models know numbers? probing numeracy in embeddings. Presented at: 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP); Nov 3-7, 2019; Hong Kong, China. URL: https:\/\/www.aclweb.org\/anthology\/D19-1 [Accessed 12-08-2026] [doi: 10.18653\/v1\/D19-1534]","DOI":"10.18653\/v1\/D19-1534"},{"key":"R108","doi-asserted-by":"crossref","unstructured":"Cheng F Li H Liu F van Rooij R Zhang K Lin Z . Empowering LLMs with logical reasoning: a comprehensive survey. Presented at: Proceedings of the Thirty-Fourth International Joint Conference on Artificial Intelligence; Aug 16-22, 2025; Montreal, Canada. URL: https:\/\/www.ijcai.org\/proceedings\/2025 [Accessed 20-09-2026] [doi: 10.24963\/ijcai.2025\/1155]","DOI":"10.24963\/ijcai.2025\/1155"},{"key":"R109","doi-asserted-by":"crossref","unstructured":"Zamaraeva O Flickinger D Bond F G\u00f3mez-Rodr\u00edguez C . Comparing LLM-generated and human-authored news text using formal syntactic theory. Presented at: 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers); Jul 27 to Aug 1, 2025; Vienna, Austria. URL: https:\/\/aclanthology.org\/2025.acl-long [Accessed 12-08-2026] [doi: 10.18653\/v1\/2025.acl-long.443]","DOI":"10.18653\/v1\/2025.acl-long.443"},{"key":"R110","doi-asserted-by":"crossref","unstructured":"Liu S Yu Z Huang F Bulbulia Y Bergen A Liut M . Can small language models with retrieval-augmented generation replace large language models when learning computer science? Presented at: 2024 on Innovation and Technology in Computer Science Education V 1 (ITiCSE 2024); Jul 8-10, 2024:388-393; Milan, Italy. URL: https:\/\/dl.acm.org\/doi\/proceedings\/10.1145\/3649217 [Accessed 12-08-2026] [doi: 10.1145\/3649217.3653554]","DOI":"10.1145\/3649217.3653554"},{"key":"R111","unstructured":"Shandilya B Palmer A . Boosting the capabilities of compact models in low-data contexts with large language models and retrieval-augmented generation. Presented at: 31st International Conference on Computational Linguistics; Jan 21-24, 2025; Abu Dhabi, UAE. URL: https:\/\/aclanthology.org\/2025.coling-main.499\/ [Accessed 06-09-2026]"},{"issue":"3","key":"R112","doi-asserted-by":"publisher","first-page":"89","DOI":"10.3390\/make7030089","article-title":"AlzheimerRAG: multimodal retrieval-augmented generation for clinical use cases","volume":"7","author":"Lahiri","journal-title":"Mach Learn Knowl Extr"},{"issue":"10","key":"R113","doi-asserted-by":"publisher","first-page":"6509","DOI":"10.1097\/JS9.0000000000001850","article-title":"The performance of large language model-powered chatbots compared to oncology physicians on colorectal cancer queries","volume":"110","author":"Zhou","journal-title":"Int J Surg"},{"key":"R114","doi-asserted-by":"publisher","DOI":"10.1148\/ryai.240476","article-title":"RadioRAG: factual large language models for enhanced diagnostics in radiology using online retrieval augmented generation","author":"Arasteh","journal-title":"Radiol Artif Intell"},{"issue":"3","key":"R115","doi-asserted-by":"publisher","first-page":"3713","DOI":"10.1007\/s11042-022-13428-4s","article-title":"Natural language processing: state of the art, current trends and challenges","volume":"82","author":"Khurana","journal-title":"Multimed Tools Appl"},{"key":"R116","doi-asserted-by":"crossref","unstructured":"Jabarulla MY Oeltze-Jafra S Beerbaum P Uden T . MedDoc-Bot: a chat tool for comparative analysis of large language models in the context of the pediatric hypertension guideline. Presented at: 2024 46th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC); Jul 15-19, 2024:1-4; Orlando, FL, USA. URL: https:\/\/ieeexplore.ieee.org\/document\/10781509 [Accessed 20-09-2026] [doi: 10.1109\/EMBC53108.2024.10781509]","DOI":"10.1109\/EMBC53108.2024.10781509"},{"issue":"1","key":"R117","first-page":"52","volume":"2","author":"Aghaziarati","journal-title":"J Foresight Health Gov"},{"key":"R118","doi-asserted-by":"publisher","DOI":"10.3389\/fdgth.2025.1606291","article-title":"Balancing risks and benefits: clinicians\u2019 perspectives on the use of generative AI chatbots in mental healthcare","volume":"7","author":"Hipgrave","journal-title":"Front Digit Health"},{"issue":"9","key":"R119","doi-asserted-by":"publisher","first-page":"115","DOI":"10.3390\/bdcc8090115","article-title":"QA-RAG: exploring LLM reliance on external knowledge","volume":"8","author":"Mansurova","journal-title":"Big Data Cogn Comput"},{"issue":"4","key":"R120","doi-asserted-by":"publisher","first-page":"335","DOI":"10.1007\/s11904-019-00455-4","article-title":"Design considerations for implementing eHealth behavioral interventions for HIV prevention in evolving sociotechnical landscapes","volume":"16","author":"Li","journal-title":"Curr HIV\/AIDS Rep"},{"issue":"5","key":"R121","doi-asserted-by":"publisher","DOI":"10.2196\/jmir.7725","article-title":"Accelerating digital mental health research from early design and creation to successful implementation and sustainment","volume":"19","author":"Mohr","journal-title":"J Med Internet Res"}],"container-title":["JMIR Research Protocols"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,10,1]],"date-time":"2026-10-01T20:13:02Z","timestamp":1790885582000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.researchprotocols.org\/2026\/1\/e79966"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,1]]},"references-count":121,"URL":"https:\/\/doi.org\/10.2196\/79966","relation":{},"ISSN":["1929-0748"],"issn-type":[{"value":"1929-0748","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,1]]},"article-number":"v15i6e79966"}}