{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,11]],"date-time":"2026-10-11T17:46:28Z","timestamp":1791740788925,"version":"4.3.4"},"reference-count":19,"publisher":"SAGE Publications","license":[{"start":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T00:00:00Z","timestamp":1791417600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"name":"Center for Advancing Safety of Machine Intelligence"},{"DOI":"10.13039\/100000083","name":"Directorate for Computer and Information Science and Engineering","doi-asserted-by":"publisher","award":["2219586"],"award-info":[{"award-number":["2219586"]}],"id":[{"id":"10.13039\/100000083","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["Health Educ Behav"],"abstract":"<jats:p>Conversational intelligent tutoring systems (CITS) have the potential to bring health education to low-income communities at scale. Although educational attainment is usually low, the health literacy spectrum within these communities can be wide. Therefore, special attention should be paid when designing and evaluating CITS within these communities. Visual, linguistic, subject matter, and cultural models must be considered. Moreover, evaluating CITS can be challenging. The instruments used to assess the impact of these systems must consider the levels of health literacy of the individuals who interact with them. In this work, we present a community-engaged design process of a breast cancer CITS for Latinas from low-income communities in Chicago. The system comprises an autonomous version and a wizard-of-Oz version\u2014a system that appears to work autonomously to its users but is operated remotely by researchers to control for potential flaws in its artificial intelligence engine. Then, we present and contrast a case study of two women who interacted with the system. Although the two women in this paper would be traditionally grouped together for statistical analysis, the evaluations we obtained suggest that for one of them, the evaluation instruments do not seem to accurately capture her experiences interacting with the CITS. We conclude by highlighting the need for purposefully designed systems for low-income communities that rely on cultural models as well as language models and advocate for designing evaluation instruments that faithfully capture the experiences of all participants.<\/jats:p>","DOI":"10.1177\/10901981261490851","type":"journal-article","created":{"date-parts":[[2026,10,8]],"date-time":"2026-10-08T12:55:26Z","timestamp":1791464126000},"update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":0,"title":["The Challenges of Designing Intelligent Tutoring Systems for Health Care in Low-Income Latina Populations"],"prefix":"10.1177","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0876-481X","authenticated-orcid":false,"given":"Francisco","family":"Iacobelli","sequence":"first","affiliation":[{"name":"Loyola University Chicago, IL, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7117-3716","authenticated-orcid":false,"given":"Chantal","family":"Reyna","sequence":"additional","affiliation":[{"name":"Loyola University Chicago, IL, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Abena","family":"Antobre","sequence":"additional","affiliation":[{"name":"Loyola University Chicago, IL, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nathaly","family":"Gonzalez","sequence":"additional","affiliation":[{"name":"Loyola University Chicago, IL, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Cordelia","family":"De La Fuente","sequence":"additional","affiliation":[{"name":"Loyola University Chicago, IL, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1393-2416","authenticated-orcid":false,"given":"Renu","family":"Balyan","sequence":"additional","affiliation":[{"name":"SUNY Old Westbury, Long Island, NY, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2026,10,8]]},"reference":[{"key":"e_1_3_2_2_1","doi-asserted-by":"crossref","unstructured":"Abbasian M. Azimi I. Rahmani A. M. Jain R. (2025). Conversational health agents: A personalized large language model-powered agent framework. JAMIA Open 8(4) ooaf067. https:\/\/doi.org\/10.1093\/jamiaopen\/ooaf067","DOI":"10.1093\/jamiaopen\/ooaf067"},{"key":"e_1_3_2_3_1","doi-asserted-by":"crossref","unstructured":"Adler N. E. Epel E. S. Castellazzo G. Ickovics J. R. (2000). Relationship of subjective and objective social status with psychological and physiological functioning: Preliminary data in healthy White women. Health Psychology 19(6) 586\u2013592. https:\/\/doi.org\/10.1037\/0278-6133.19.6.586","DOI":"10.1037\/0278-6133.19.6.586"},{"key":"e_1_3_2_4_1","doi-asserted-by":"crossref","unstructured":"Bartneck C. Kuli\u0107 D. Croft E. Zoghbi S. (2009). Measurement instruments for the anthropomorphism animacy likeability perceived intelligence and perceived safety of robots. International Journal of Social Robotics 1(1) 71\u201381. http:\/\/www.bartneck.de\/publications\/2009\/measurementInstrumentsRobots","DOI":"10.1007\/s12369-008-0001-3"},{"key":"e_1_3_2_5_1","doi-asserted-by":"crossref","unstructured":"Brooke J. (1996). SUS\u2014A Quick and Dirty Usability Scale. Usability Evaluation in Industry 189(194) 4\u20137. https:\/\/doi.org\/10.1002\/hbm.20701","DOI":"10.1002\/hbm.20701"},{"key":"e_1_3_2_6_1","doi-asserted-by":"crossref","unstructured":"Christy S. M. Cousin L. A. Sutton S. K. Chavarria E. A. Abdulla R. Gutierrez L. Sanchez J. Lopez D. Gwede C. K. Meade C. D. (2021). Characterizing health literacy among Spanish language-preferring Latinos ages 50\u201375. Nursing Research 70(5) 344\u2013353. https:\/\/doi.org\/10.1097\/NNR.0000000000000519","DOI":"10.1097\/NNR.0000000000000519"},{"key":"e_1_3_2_7_1","doi-asserted-by":"crossref","unstructured":"Fang Y. Lippert A. Cai Z. Chen S. Frijters J. C. Greenberg D. Graesser A. C. (2021). Patterns of adults with low literacy skills interacting with an intelligent tutoring system. International Journal of Artificial Intelligence in Education 32 297\u2013322. https:\/\/link.springer.com\/10.1007\/s40593-021-00266-y","DOI":"10.1007\/s40593-021-00266-y"},{"key":"e_1_3_2_8_1","doi-asserted-by":"crossref","unstructured":"Hahn E. A. Choi S. W. Griffith J. W. Yost K. J. Baker D. W. (2011). Health literacy assessment using talking touchscreen technology (Health LiTT): A new item response theory-based measure of health literacy. Journal of Health Communication 16(Suppl. 3) 150\u2013162. http:\/\/www.ncbi.nlm.nih.gov\/pubmed\/21951249","DOI":"10.1080\/10810730.2011.605434"},{"key":"e_1_3_2_9_1","doi-asserted-by":"crossref","unstructured":"Harrison R. Flood D. Duce D. (2013). Usability of mobile applications: Literature review and rationale for a new usability model. Journal of Interaction Science 1(1) 1\u201316. https:\/\/doi.org\/10.1186\/2194-0827-1-1","DOI":"10.1186\/2194-0827-1-1"},{"key":"e_1_3_2_10_1","first-page":"207","volume-title":"Designing for diversity: Multicultural characters for a multi-cultural world","author":"Hayes-Roth B.","year":"2002","unstructured":"Hayes-Roth B., Maldonado H., Moraes M. (2002, February 12\u201315). Designing for diversity: Multicultural characters for a multi-cultural world [Conference Proceedings]. Proceedings of Imagina (pp. 207\u2013225), Monte Carlo, Monaco."},{"key":"e_1_3_2_11_1","doi-asserted-by":"crossref","unstructured":"Huo B. Boyle A. Marfo N. Tangamornsuksan W. Steen J. P. McKechnie T. Lee Y. Mayol J. Antoniou S. A. Thirunavukarasu A. J. Sanger S. Ramji K. Guyatt G. (2025). Large language models for chatbot health advice studies. JAMA Network Open 8(2) e2457879. https:\/\/doi.org\/10.1001\/jamanetworkopen.2024.57879","DOI":"10.1001\/jamanetworkopen.2024.57879"},{"key":"e_1_3_2_12_1","doi-asserted-by":"crossref","unstructured":"Iacobelli F. Adler R. F. Buitrago D. Buscemi J. Corden M. Perez-Tamayo A. Penedo F. Rodriguez M. Yanez B. (2018). Designing an mHealth application to bridge health disparities in Latina breast cancer survivors: a community-supported design approach. Design for Health 2(1) 58\u201376. https:\/\/doi.org\/10.1080\/24735132.2018.1452871","DOI":"10.1080\/24735132.2018.1452871"},{"key":"e_1_3_2_13_1","doi-asserted-by":"crossref","unstructured":"Iacobelli F. Dragon G. Mazur G. Guitelman J. (2021). Web-based information seeking behaviors of low-literacy Hispanic survivors of breast cancer: Observational pilot study. JMIR Form Res 5(10) e22809. https:\/\/doi.org\/10.2196\/22809","DOI":"10.2196\/22809"},{"key":"e_1_3_2_14_1","doi-asserted-by":"crossref","unstructured":"Jemmali M. Dragon G. Mendoza R. Stille D. Poinar P. Ojha A. Baez K. Jeong S. Balyan R. Iacobelli F. (2024 November 9\u201313). Sorting pauses in speech based dialog systems: Effects of knowing whether pauses in speech are mid-turn or at the end of the turn [Conference Proceedings]. Companion publication of the 2024 conference on computer-supported cooperative work and social computing (pp. 587\u2013591) San Jose Costa Rica. ACM. https:\/\/doi.org\/10.1145\/3678884.3681910","DOI":"10.1145\/3678884.3681910"},{"key":"e_1_3_2_15_1","doi-asserted-by":"crossref","unstructured":"Ji Y. Ma W. Sivarajkumar S. Zhang H. Sadhu E. M. Li Z. Wang Y. (2025). Mitigating the risk of health inequity exacerbated by large language models. NPJ Digital Medicine 8(1) 246. https:\/\/doi.org\/10.1038\/s41746-025-01576-4","DOI":"10.1038\/s41746-025-01576-4"},{"key":"e_1_3_2_16_1","doi-asserted-by":"crossref","unstructured":"Lin S.-J. Sun C.-Y. Chen D.-N. Kang Y.-N. Hoang K. D. Chen K.-H. Chen C. (2025). Chatbots for breast cancer education: A systematic review and meta-analysis. Supportive Care in Cancer 33(1) 55. https:\/\/doi.org\/10.1007\/s00520-024-09096-9","DOI":"10.1007\/s00520-024-09096-9"},{"key":"e_1_3_2_17_1","doi-asserted-by":"publisher","DOI":"10.17730\/1938-3525-81.4.327"},{"key":"e_1_3_2_18_1","doi-asserted-by":"crossref","unstructured":"Shaban M. Osman Y. M. Mohamed N. A. Shaban M. M. (2025). Empowering breast cancer clients through AI chatbots: Transforming knowledge and attitudes for enhanced nursing care. BMC Nursing 24(1) 994. https:\/\/doi.org\/10.1186\/s12912-025-03585-w","DOI":"10.1186\/s12912-025-03585-w"},{"key":"e_1_3_2_19_1","doi-asserted-by":"crossref","unstructured":"Wei J. Kim S. Jung H. Kim Y.-H. (2024). Leveraging large language models to power chatbots for collecting user self-reported data. Proceedings of the ACM on Human\u2013Computer Interaction 8(CSCW1) 1\u201335. https:\/\/doi.org\/10.1145\/3637364","DOI":"10.1145\/3637364"},{"key":"e_1_3_2_20_1","doi-asserted-by":"crossref","unstructured":"Yanez B. R. Buitrago D. Buscemi J. Iacobelli F. Adler R. F. Corden M. E. Perez-Tamayo A. Guitelman J. Penedo F. J. (2018). Study design and protocol for My Guide: An e-health intervention to improve patient-centered outcomes among Hispanic breast cancer survivors. Contemporary Clinical Trials 65 61\u201368. http:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S1551714417305396doi:10.1016\/j.cct.2017.11.018","DOI":"10.1016\/j.cct.2017.11.018"}],"container-title":["Health Education &amp; Behavior"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/10901981261490851","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/10901981261490851","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/10901981261490851","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,10,11]],"date-time":"2026-10-11T17:06:51Z","timestamp":1791738411000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/10901981261490851"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,8]]},"references-count":19,"alternative-id":["10.1177\/10901981261490851"],"URL":"https:\/\/doi.org\/10.1177\/10901981261490851","relation":{},"ISSN":["1090-1981","1552-6127"],"issn-type":[{"value":"1090-1981","type":"print"},{"value":"1552-6127","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,8]]},"article-number":"10901981261490851"}}