{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,6]],"date-time":"2026-11-06T13:59:42Z","timestamp":1793973582857,"version":"4.8.0"},"reference-count":18,"publisher":"SAGE Publications","issue":"5","license":[{"start":{"date-parts":[[2024,5,20]],"date-time":"2024-05-20T00:00:00Z","timestamp":1716163200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/journals.sagepub.com\/page\/policies\/text-and-data-mining-license"}],"funder":[{"DOI":"10.13039\/100000062","name":"National Institute of Diabetes and Digestive and Kidney Diseases","doi-asserted-by":"publisher","award":["1R01124789-01A1"],"award-info":[{"award-number":["1R01124789-01A1"]}],"id":[{"id":"10.13039\/100000062","id-type":"DOI","asserted-by":"publisher"}]},{"name":"National Science Foundation Center to Stream Healthcare in Place","award":["2052578"],"award-info":[{"award-number":["2052578"]}]},{"name":"Silverberry Group, Inc."}],"content-domain":{"domain":["journals.sagepub.com"],"crossmark-restriction":true},"short-container-title":["J Diabetes Sci Technol"],"published-print":{"date-parts":[[2025,9]]},"abstract":"<jats:sec>\n                    <jats:title>Background:<\/jats:title>\n                    <jats:p>Large language models (LLMs) offer significant potential in medical information extraction but carry risks of generating incorrect information. This study aims to develop and validate a retriever-augmented generation (RAG) model that provides accurate medical knowledge about diabetes and diabetic foot care to laypersons with an eighth-grade literacy level. Improving health literacy through patient education is paramount to addressing the problem of limb loss in the diabetic population. In addition to affecting patient well-being through improved outcomes, improved physician well-being is an important outcome of a self-management model for patient health education.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods:<\/jats:title>\n                    <jats:p>We used an RAG architecture and built a question-and-answer artificial intelligence (AI) model to extract knowledge in response to questions pertaining to diabetes and diabetic foot care. We utilized GPT-4 by OpenAI, with Pinecone as a vector database. The NIH National Standards for Diabetes Self-Management Education served as the basis for our knowledge base. The model\u2019s outputs were validated through expert review against established guidelines and literature. Fifty-eight keywords were used to select 295 articles and the model was tested against 175 questions across topics.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results:<\/jats:title>\n                    <jats:p>The study demonstrated that with appropriate content volume and few-shot learning prompts, the RAG model achieved 98% accuracy, confirming its capability to offer user-friendly and comprehensible medical information.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusion:<\/jats:title>\n                    <jats:p>The RAG model represents a promising tool for delivering reliable medical knowledge to the public which can be used for self-education and self-management for diabetes, highlighting the importance of content validation and innovative prompt engineering in AI applications.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1177\/19322968241253568","type":"journal-article","created":{"date-parts":[[2024,5,20]],"date-time":"2024-05-20T09:08:48Z","timestamp":1716196128000},"page":"1264-1270","update-policy":"https:\/\/doi.org\/10.1177\/sage-journals-update-policy","source":"Crossref","is-referenced-by-count":32,"title":["Building Trustworthy Generative Artificial Intelligence for Diabetes Care and Limb Preservation: A Medical Knowledge Extraction Case"],"prefix":"10.1177","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-2131-3367","authenticated-orcid":false,"given":"Shayan","family":"Mashatian","sequence":"first","affiliation":[{"name":"Biomedical Engineering Program, University of North Dakota, Grand Forks, ND, USA"},{"name":"Silverberry Group, Inc., Grand Forks, ND, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"David G.","family":"Armstrong","sequence":"additional","affiliation":[{"name":"Keck School of Medicine, University of Southern California, Los Angeles, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Aaron","family":"Ritter","sequence":"additional","affiliation":[{"name":"U.S. Department of Veterans Affairs, Little Rock, AR, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jeffery","family":"Robbins","sequence":"additional","affiliation":[{"name":"U.S. Department of Veterans Affairs, Washington, DC, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Shereen","family":"Aziz","sequence":"additional","affiliation":[{"name":"Silverberry Group, Inc., Grand Forks, ND, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Ilia","family":"Alenabi","sequence":"additional","affiliation":[{"name":"Silverberry Group, Inc., Grand Forks, ND, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Michelle","family":"Huo","sequence":"additional","affiliation":[{"name":"Department of Life Sciences, University of California, Los Angeles, Los Angeles, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Taneeka","family":"Anand","sequence":"additional","affiliation":[{"name":"Department of Ecology and Evolutionary Biology, University of California, Los Angeles, Los Angeles, CA, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kouhyar","family":"Tavakolian","sequence":"additional","affiliation":[{"name":"Biomedical Engineering Program, University of North Dakota, Grand Forks, ND, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"179","published-online":{"date-parts":[[2024,5,20]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.iswa.2023.200308."},{"key":"e_1_3_3_3_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.heliyon.2023.e23050."},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-023-06291-2."},{"key":"e_1_3_3_5_2","article-title":"Retrieval-augmented generation for knowledge-intensive NLP tasks","author":"Lweis P","year":"2021","unstructured":"Lweis P. Retrieval-augmented generation for knowledge-intensive NLP tasks. arXiv:2005.11401v4 [cs.CL]. 2021.","journal-title":"arXiv:2005.11401v4 [cs.CL]"},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.2337\/dci22-0043."},{"key":"e_1_3_3_7_2","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2023.10578."},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.1186\/s13047-020-00383-2"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.2337\/dc15-0536"},{"key":"e_1_3_3_10_2","doi-asserted-by":"publisher","DOI":"10.3390\/jcm9061780"},{"key":"e_1_3_3_11_2","unstructured":"International Diabetes Federation. The Diabetic Foot. Brussels Belgium: International Diabetes Federation; 2020. https:\/\/www.idf.org\/our-activities\/care-prevention\/diabetic-foot.html. Accessed August 1 2022."},{"key":"e_1_3_3_12_2","unstructured":"Armstrong DG Mills JL Molina M Molnar JA. Nutrition Interventions in Adults With Diabetic Foot Ulcers. Guideline Central; 2021. https:\/\/www.guidelinecentral.com\/guideline\/502765\/pocket-guide\/502768\/. Accessed June 26 2022."},{"key":"e_1_3_3_13_2","doi-asserted-by":"crossref","unstructured":"Donnelly HR Clarke ED Collins CE et al. Most individuals with diabetes-related foot ulceration do not meet dietary consensus guidelines for wound healing. Int Wound J. 2023;21:e14483. https:\/\/onlinelibrary.wiley.com\/doi\/abs\/10.1111\/iwj.14483. Accessed November 11 2023.","DOI":"10.1111\/iwj.14483"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1177\/014107680309600504."},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.1080\/08998280.2017.11929552."},{"key":"e_1_3_3_16_2","doi-asserted-by":"publisher","DOI":"10.1001\/jama.2018.1331."},{"key":"e_1_3_3_17_2","doi-asserted-by":"crossref","unstructured":"Timmers T Janssen L Kool R Kremer J. Educating patients by providing timely information using smartphone and tablet apps: systematic review. J Med Internet Res. 2020;22(4):e17342. https:\/\/www.jmir.org\/2020\/4\/e17342. Accessed April 30 2024.","DOI":"10.2196\/17342"},{"key":"e_1_3_3_18_2","unstructured":"American Medical Association. 6 big things that must change to beat physician burnout. https:\/\/www.ama-assn.org\/practice-management\/physician-health\/6-big-things-must-change-beat-physician-burnout. Accessed April 30 2024."},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.2337\/dc10-S089."}],"container-title":["Journal of Diabetes Science and Technology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/19322968241253568","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/full-xml\/10.1177\/19322968241253568","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/journals.sagepub.com\/doi\/pdf\/10.1177\/19322968241253568","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T18:17:01Z","timestamp":1777400221000},"score":1,"resource":{"primary":{"URL":"https:\/\/journals.sagepub.com\/doi\/10.1177\/19322968241253568"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,5,20]]},"references-count":18,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2025,9]]}},"alternative-id":["10.1177\/19322968241253568"],"URL":"https:\/\/doi.org\/10.1177\/19322968241253568","relation":{},"ISSN":["1932-2968","1932-2968"],"issn-type":[{"value":"1932-2968","type":"print"},{"value":"1932-2968","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,5,20]]}}}