{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,9]],"date-time":"2026-06-09T04:45:17Z","timestamp":1780980317007,"version":"3.54.1"},"reference-count":34,"publisher":"Public Library of Science (PLoS)","issue":"1","license":[{"start":{"date-parts":[[2025,1,15]],"date-time":"2025-01-15T00:00:00Z","timestamp":1736899200000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100000024","name":"Canadian Institutes of Health Research","doi-asserted-by":"publisher","award":["440231"],"award-info":[{"award-number":["440231"]}],"id":[{"id":"10.13039\/501100000024","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004828","name":"Grand Challenges Canada","doi-asserted-by":"publisher","award":["TTS-2003-37605"],"award-info":[{"award-number":["TTS-2003-37605"]}],"id":[{"id":"10.13039\/501100004828","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["www.plosdigitalhealth.org"],"crossmark-restriction":false},"short-container-title":["PLOS Digit Health"],"abstract":"<jats:p>Community isolation of patients with communicable infectious diseases limits spread of pathogens but our understanding of isolated patients\u2019 needs and challenges is incomplete. Rwanda deployed a digital health service nationally to assist public health clinicians to remotely monitor and support SARS-CoV-2 cases via their mobile phones using daily interactive short message service (SMS) check-ins. We aimed to assess the texting patterns and communicated topics to better understand patient experiences. We extracted data on all COVID-19 cases and exposed contacts who were enrolled in the WelTel text messaging program between March 18, 2020, and March 31, 2022, and linked demographic and clinical data from the national COVID-19 registry. A sample of the text conversation corpus was English-translated and labeled with topics of interest defined by medical experts. Multiple natural language processing (NLP) topic classification models were trained and compared using F1 scores. Best performing models were applied to classify unlabeled conversations. Total 33,081 isolated patients (mean age 33\u00b79, range 0\u2013100), 44% female, including 30,398 cases and 2,683 contacts) were registered in WelTel. Registered patients generated 12,119 interactive text conversations in Kinyarwanda (n = 8,183, 67%), English (n = 3,069, 25%) and other languages. Sufficiently trained large language models (LLMs) were unavailable for Kinyarwanda. Traditional machine learning (ML) models outperformed fine-tuned transformer architecture language models on the native untranslated language corpus, however, the reverse was observed of models trained on English-only data. The most frequently identified topics discussed included symptoms (69%), diagnostics (38%), social issues (19%), prevention (18%), healthcare logistics (16%), and treatment (8\u00b75%). Education, advice, and triage on these topics were provided to patients. Interactive text messaging can be used to remotely support isolated patients in pandemics at scale. NLP can help evaluate the medical and social factors that affect isolated patients which could ultimately inform precision public health responses to future pandemics.<\/jats:p>","DOI":"10.1371\/journal.pdig.0000625","type":"journal-article","created":{"date-parts":[[2025,1,15]],"date-time":"2025-01-15T13:26:48Z","timestamp":1736947608000},"page":"e0000625","update-policy":"https:\/\/doi.org\/10.1371\/journal.pdig.corrections_policy","source":"Crossref","is-referenced-by-count":2,"title":["Natural language processing to evaluate texting conversations between patients and healthcare providers during COVID-19 Home-Based Care in Rwanda at scale"],"prefix":"10.1371","volume":"4","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1759-2236","authenticated-orcid":true,"given":"Richard T.","family":"Lester","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-6779-3766","authenticated-orcid":true,"given":"Matthew","family":"Manson","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Muhammed","family":"Semakula","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hyeju","family":"Jang","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hassan","family":"Mugabo","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ali","family":"Magzari","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Junhong Ma","family":"Blackmer","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Fanan","family":"Fattah","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Simon Pierre","family":"Niyonsenga","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Edson","family":"Rwagasore","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Charles","family":"Ruranga","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Eric","family":"Remera","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jean Claude S.","family":"Ngabonziza","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Giuseppe","family":"Carenini","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sabin","family":"Nsanzimana","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"340","published-online":{"date-parts":[[2025,1,15]]},"reference":[{"issue":"2","key":"pdig.0000625.ref001","doi-asserted-by":"crossref","first-page":"e120","DOI":"10.1016\/S2214-109X(20)30452-6","article-title":"Cost-effectiveness of public health strategies for COVID-19 epidemic control in South Africa: a microsimulation modelling study","volume":"9","author":"KP Reddy","year":"2021","journal-title":"Lancet Glob Health"},{"key":"pdig.0000625.ref002","doi-asserted-by":"crossref","unstructured":"Mbunge E, Fashoto S, Akinnuwesi B, Gurajena C, Metfula A. Challenges of Social Distancing and Self-Isolation during COVID-19 Pandemic in Africa: A Critical Review [Internet]. Rochester, NY; 2020 [cited 2024 Jun 13]. Available from: https:\/\/papers.ssrn.com\/abstract=3740202","DOI":"10.2139\/ssrn.3740202"},{"issue":"8","key":"pdig.0000625.ref003","doi-asserted-by":"crossref","first-page":"1307","DOI":"10.1038\/s41562-023-01617-6","article-title":"A systematic review and meta-analysis of 90 cohort studies of social isolation, loneliness and mortality.","volume":"7","author":"F Wang","year":"2023","journal-title":"Nat Hum Behav."},{"key":"pdig.0000625.ref004","doi-asserted-by":"crossref","first-page":"1071790","DOI":"10.3389\/fdgth.2022.1071790","article-title":"SMS-based digital health intervention in Rwanda\u2019s home-based care program for remote management of COVID-19 cases and contacts: A qualitative study of sustainability and scalability.","volume":"4","author":"A Babili","year":"2023","journal-title":"Front Digit Health.Jan 9"},{"issue":"9755","key":"pdig.0000625.ref005","doi-asserted-by":"crossref","first-page":"1838","DOI":"10.1016\/S0140-6736(10)61997-6","article-title":"Effects of a mobile phone short message service on antiretroviral treatment adherence in Kenya (WelTel Kenya1): a randomised trial.","volume":"376","author":"RT Lester","year":"2010","journal-title":"The Lancet"},{"issue":"3","key":"pdig.0000625.ref006","doi-asserted-by":"crossref","first-page":"e14219","DOI":"10.1111\/petr.14219","article-title":"A text messaging intervention and quality of life in adolescents with solid organ transplants.","volume":"26","author":"SL Rathgeber","year":"2022","journal-title":"Pediatr Transplant."},{"issue":"3","key":"pdig.0000625.ref007","doi-asserted-by":"crossref","first-page":"e143","DOI":"10.1016\/S2468-2667(17)30239-6","article-title":"Effect of an interactive text-messaging service on patient retention during the first year of HIV care in Kenya (WelTel Retain): an open-label, randomised parallel-group study.","volume":"3","author":"ML van der Kop","year":"2018","journal-title":"Lancet Public Health"},{"key":"pdig.0000625.ref008","unstructured":"MAIN INDICATORS: 5th Rwanda Population and Housing Census (PHC) | National Institute of Statistics Rwanda [Internet]. [cited 2024 Jun 13]. Available from: https:\/\/statistics.gov.rw\/publication\/main_indicators_2022"},{"issue":"1","key":"pdig.0000625.ref009","doi-asserted-by":"crossref","first-page":"3","DOI":"10.4269\/ajtmh.20-0506","article-title":"Mobile Health Technology for Enhancing the COVID-19 Response in Africa: A Potential Game Changer","volume":"103","author":"JB Nachega","year":"2020","journal-title":"Am J Trop Med Hyg"},{"issue":"9","key":"pdig.0000625.ref010","doi-asserted-by":"crossref","first-page":"e46033","DOI":"10.1371\/journal.pone.0046033","article-title":"In-Depth Analysis of Patient-Clinician Cell Phone Communication during the WelTel Kenya1 Antiretroviral Adherence Trial.","volume":"7","author":"ML van der Kop","year":"2012","journal-title":"PLOS ONE."},{"key":"pdig.0000625.ref011","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1109\/RBME.2022.3210270","article-title":"Natural Language Processing for Smart Healthcare.","volume":"17","author":"B Zhou","year":"2024","journal-title":"IEEE Rev Biomed Eng."},{"key":"pdig.0000625.ref012","article-title":"ConVIScope: Visual Analytics for Exploring Patient Conversations [Internet].","author":"R Li","year":"2021","journal-title":"arXiv"},{"issue":"7","key":"pdig.0000625.ref013","doi-asserted-by":"crossref","first-page":"e2322299","DOI":"10.1001\/jamanetworkopen.2023.22299","article-title":"Use of Natural Language Processing of Patient-Initiated Electronic Health Record Messages to Identify Patients With COVID-19","volume":"6","author":"K Mermin-Bunnell","year":"2023","journal-title":"Infection. JAMA Netw Open"},{"issue":"1","key":"pdig.0000625.ref014","doi-asserted-by":"crossref","first-page":"181","DOI":"10.1186\/s12874-022-01665-y","article-title":"A comparative study on deep learning models for text classification of unstructured medical notes with various levels of class imbalance.","volume":"22","author":"H Lu","year":"2022","journal-title":"BMC Med Res Methodol."},{"key":"pdig.0000625.ref015","doi-asserted-by":"crossref","first-page":"110","DOI":"10.1016\/j.ijmedinf.2017.06.004","article-title":"A comparison of rule-based and machine learning approaches for classifying patient portal messages","volume":"105","author":"RM Cronin","year":"2017","journal-title":"Int J Med Inf"},{"issue":"6","key":"pdig.0000625.ref016","article-title":"A Global Health Data Divide.","volume":"1","author":"X Liu","year":"2024","journal-title":"NEJM AI"},{"issue":"7809","key":"pdig.0000625.ref017","doi-asserted-by":"crossref","first-page":"384","DOI":"10.1038\/d41586-020-01563-7","article-title":"Learn from Rwanda\u2019s success in tackling COVID-19","volume":"581","author":"J Condo","year":"2020","journal-title":"Nature"},{"key":"pdig.0000625.ref018","unstructured":"Krafft M. DHIS2. 2020 [cited 2024 Jun 13]. Going paperless for COVID-19 testing in Rwanda with DHIS2 Android Capture App. Available from: https:\/\/dhis2.org\/rwanda-covid-testing\/"},{"key":"pdig.0000625.ref019","article-title":"Adaptive Machine Translation with Large Language Models [Internet].","author":"Y Moslem","year":"2023","journal-title":"arXiv"},{"issue":"10","key":"pdig.0000625.ref020","first-page":"281","article-title":"Random Search for Hyper-Parameter Optimization.","volume":"13","author":"J Bergstra","year":"2012","journal-title":"J Mach Learn Res."},{"key":"pdig.0000625.ref021","doi-asserted-by":"crossref","unstructured":"Ogueji K, Zhu Y, Lin J. Small Data? No Problem! Exploring the Viability of Pretrained Multilingual Language Models for Low-resourced Languages. In: Ataman D, Birch A, Conneau A, Firat O, Ruder S, Sahin GG, editors. Proceedings of the 1st Workshop on Multilingual Representation Learning [Internet]. Punta Cana, Dominican Republic: Association for Computational Linguistics; 2021 [cited 2024 Jun 13]. p. 116\u201326. Available from: https:\/\/aclanthology.org\/2021.mrl-1.11","DOI":"10.18653\/v1\/2021.mrl-1.11"},{"key":"pdig.0000625.ref022","article-title":"AfroLM: A Self-Active Learning-based Multilingual Pretrained Language Model for 23 African Languages [Internet].","author":"BFP Dossou","year":"2022","journal-title":"arXiv"},{"key":"pdig.0000625.ref023","article-title":"Adapting Pre-trained Language Models to African Languages via Multilingual Adaptive Fine-Tuning [Internet].","author":"JO Alabi","year":"2022","journal-title":"arXiv"},{"key":"pdig.0000625.ref024","unstructured":"jean-paul\/KinyaBERT-small \u00b7 Hugging Face [Internet]. [cited 2024 Jun 13]. Available from: https:\/\/huggingface.co\/jean-paul\/KinyaBERT-small"},{"key":"pdig.0000625.ref025","article-title":"BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding [Internet].","author":"J Devlin","year":"2019","journal-title":"arXiv"},{"key":"pdig.0000625.ref026","article-title":"Longformer: The Long-Document Transformer [Internet].","author":"I Beltagy","year":"2020","journal-title":"arXiv"},{"key":"pdig.0000625.ref027","unstructured":"Rwanda\u2019s literacy rate rises | National Institute of Statistics Rwanda [Internet]. [cited 2024 Jun 13]. Available from: https:\/\/statistics.gov.rw\/publications\/article\/rwanda%E2%80%99s-literacy-rate-rises"},{"issue":"7","key":"pdig.0000625.ref028","first-page":"1028","article-title":"Secure Messaging and COVID-19: A Content Analysis of Patient-Clinician Communication During the Pandemic.","volume":"28","author":"JM Alpert","year":"2022","journal-title":"Telemed J E-Health Off J Am Telemed Assoc."},{"issue":"11","key":"pdig.0000625.ref029","doi-asserted-by":"crossref","first-page":"6591","DOI":"10.3390\/ijerph19116591","article-title":"Concerns of Home Isolating COVID-19 Patients While Receiving Care via Telemedicine during the Pandemic in the Northern Thailand: A Qualitative Study on Text Messaging.","volume":"19","author":"K Pinyopornpanish","year":"2022","journal-title":"Int J Environ Res Public Health"},{"issue":"6","key":"pdig.0000625.ref030","article-title":"Clinical Text Datasets for Medical Artificial Intelligence and Large Language Models\u2014A Systematic Review.","volume":"1","author":"J Wu","year":"2024","journal-title":"NEJM AI"},{"key":"pdig.0000625.ref031","article-title":"A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarios [Internet].","author":"MA Hedderich","year":"2021","journal-title":"arXiv"},{"key":"pdig.0000625.ref032","doi-asserted-by":"crossref","first-page":"1063466","DOI":"10.3389\/fpubh.2023.1063466","article-title":"Development and testing of a multi-lingual Natural Language Processing-based deep learning system in 10 languages for COVID-19 pandemic crisis: A multi-center study.","volume":"11","author":"LWY Yang","year":"2023","journal-title":"Front Public Health"},{"issue":"6","key":"pdig.0000625.ref033","doi-asserted-by":"crossref","DOI":"10.1056\/AIpc2400033","article-title":"Ethical Oversight of Data Science Health Research in Africa.","volume":"1","author":"C Adebamowo","year":"2024","journal-title":"NEJM AI"},{"issue":"1","key":"pdig.0000625.ref034","doi-asserted-by":"crossref","DOI":"10.1056\/AIpc2300038","article-title":"Development Pipeline and Geographic Representation of Trials for Artificial Intelligence\/Machine Learning\u2013Enabled Medical Devices (2010 to 2023).","volume":"1","author":"M Serra-Burriel","year":"2023","journal-title":"NEJM AI."}],"container-title":["PLOS Digital Health"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pdig.0000625","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,1,15]],"date-time":"2025-01-15T13:27:11Z","timestamp":1736947631000},"score":1,"resource":{"primary":{"URL":"https:\/\/dx.plos.org\/10.1371\/journal.pdig.0000625"}},"subtitle":[],"editor":[{"given":"Hadi","family":"Ghasemi","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"editor"}]}],"short-title":[],"issued":{"date-parts":[[2025,1,15]]},"references-count":34,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,1,15]]}},"URL":"https:\/\/doi.org\/10.1371\/journal.pdig.0000625","relation":{"has-preprint":[{"id-type":"doi","id":"10.1101\/2024.08.30.24312636","asserted-by":"object"}]},"ISSN":["2767-3170"],"issn-type":[{"value":"2767-3170","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,1,15]]}}}