{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,25]],"date-time":"2026-03-25T14:54:38Z","timestamp":1774450478671,"version":"3.50.1"},"reference-count":63,"publisher":"MDPI AG","issue":"11","license":[{"start":{"date-parts":[[2024,11,18]],"date-time":"2024-11-18T00:00:00Z","timestamp":1731888000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["BDCC"],"abstract":"<jats:p>Due to the widespread acceptance of ChatGPT, implementing large language models (LLMs) in real-world applications has become an important research area. Such productisation of technologies allows the public to use AI without technical knowledge. LLMs can revolutionise and automate various healthcare processes, but security is critical. If implemented in critical sectors such as healthcare, adversaries can manipulate the vulnerabilities present in such systems to perform malicious activities such as data exfiltration and manipulation, and the results can be devastating. While LLM implementation in healthcare has been discussed in numerous studies, threats and vulnerabilities identification in LLMs and their safe implementation in healthcare remain largely unexplored. Based on a comprehensive review, this study provides new findings which do not exist in the current literature. This research has proposed a taxonomy to explore LLM applications in healthcare, a threat model considering the vulnerabilities of LLMs which may affect their implementation in healthcare, and a security framework for the implementation of LLMs in healthcare and has identified future avenues of research in LLMs, cybersecurity, and healthcare.<\/jats:p>","DOI":"10.3390\/bdcc8110161","type":"journal-article","created":{"date-parts":[[2024,11,18]],"date-time":"2024-11-18T05:36:45Z","timestamp":1731908205000},"page":"161","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":17,"title":["A Review of Large Language Models in Healthcare: Taxonomy, Threats, Vulnerabilities, and Framework"],"prefix":"10.3390","volume":"8","author":[{"ORCID":"https:\/\/orcid.org\/0009-0001-0974-9910","authenticated-orcid":false,"given":"Rida","family":"Hamid","sequence":"first","affiliation":[{"name":"School of Computing and Creative Technologies, University of the West of England, Bristol BS16 1QY, UK"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3463-0484","authenticated-orcid":false,"given":"Sarfraz","family":"Brohi","sequence":"additional","affiliation":[{"name":"School of Computing and Creative Technologies, University of the West of England, Bristol BS16 1QY, UK"}]}],"member":"1968","published-online":{"date-parts":[[2024,11,18]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"102274","DOI":"10.1016\/j.lindif.2023.102274","article-title":"ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education","volume":"103","author":"Kasneci","year":"2023","journal-title":"Learn. Individ. Differ."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"100017","DOI":"10.1016\/j.metrad.2023.100017","article-title":"Summary of ChatGPT-Related Research and Perspective towards the Future of Large Language Models","volume":"1","author":"Liu","year":"2023","journal-title":"Meta-Radiol."},{"key":"ref_3","unstructured":"Microsoft Research (2023). Microsoft the New Bing: Our Approach to Responsible AI, Microsoft Research."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"33","DOI":"10.1007\/s10916-023-01925-4","article-title":"Evaluating the Feasibility of ChatGPT in Healthcare: An Analysis of Multiple Clinical and Research Scenarios","volume":"47","author":"Cascella","year":"2023","journal-title":"J. Med. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"990","DOI":"10.1016\/j.jacr.2023.05.003","article-title":"Evaluating GPT as an Adjunct for Radiologic Decision Making: GPT-4 Versus GPT-3.5 in a Breast Imaging Pilot","volume":"20","author":"Rao","year":"2023","journal-title":"J. Am. Coll. Radiol."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Sallam, M. (2023). ChatGPT Utility in Healthcare Education, Research, and Practice: Systematic Review on the Promising Perspectives and Valid Concerns. Healthcare, 11.","DOI":"10.3390\/healthcare11060887"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"e179","DOI":"10.1016\/S2589-7500(23)00048-1","article-title":"Using ChatGPT to Write Patient Clinic Letters","volume":"5","author":"Ali","year":"2023","journal-title":"Lancet Digit. Health"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"e107","DOI":"10.1016\/S2589-7500(23)00021-3","article-title":"ChatGPT: The Future of Discharge Summaries?","volume":"5","author":"Patel","year":"2023","journal-title":"Lancet Digit. Health"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"194","DOI":"10.1038\/s41746-022-00742-2","article-title":"A Large Language Model for Electronic Health Records","volume":"5","author":"Yang","year":"2022","journal-title":"NPJ Digit. Med."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"641","DOI":"10.1016\/S0140-6736(23)00216-7","article-title":"The Promise of Large Language Models in Health Care","volume":"401","author":"Arora","year":"2023","journal-title":"Lancet"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Straw, I., and Callison-Burch, C. (2020). Artificial Intelligence in Mental Health and the Biases of Language Based Models. PLoS ONE, 15.","DOI":"10.1371\/journal.pone.0240376"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"48","DOI":"10.1016\/j.maturitas.2018.04.008","article-title":"Cybersecurity in Healthcare: A Narrative Review of Trends, Threats and Ways Forward","volume":"113","author":"Coventry","year":"2018","journal-title":"Maturitas"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"109729","DOI":"10.1016\/j.resuscitation.2023.109729","article-title":"Exploring ChatGPT for Information of Cardiopulmonary Resuscitation","volume":"185","author":"Ahn","year":"2023","journal-title":"Resuscitation"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"663","DOI":"10.1227\/neu.0000000000002414","article-title":"I Asked a ChatGPT to Write an Editorial About How We Can Incorporate Chatbots Into Neurosurgical Research and Patient Care\u2026","volume":"92","author":"White","year":"2023","journal-title":"Neurosurgery"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"102744","DOI":"10.1016\/j.dsx.2023.102744","article-title":"ChatGPT: Is This Version Good for Healthcare and Research?","volume":"17","author":"Vaishya","year":"2023","journal-title":"Diabetes Metab. Syndr. Clin. Res. Rev."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Pan, X., Zhang, M., Ji, S., and Yang, M. (2020, January 18\u201321). Privacy Risks of General-Purpose Language Models. Proceedings of the 2020 IEEE Symposium on Security and Privacy (SP), San Francisco, CA, USA.","DOI":"10.1109\/SP40000.2020.00095"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"e003105","DOI":"10.1136\/rmdopen-2023-003105","article-title":"The Wide Range of Opportunities for Large Language Models Such as ChatGPT in Rheumatology","volume":"9","year":"2023","journal-title":"RMD Open"},{"key":"ref_18","unstructured":"Weidinger, L., Mellor, J., Rauh, M., Griffin, C., Uesato, J., Huang, P.-S., Cheng, M., Glaese, M., Balle, B., and Kasirzadeh, A. (2021). Ethical and Social Risks of Harm from Language Models. arXiv."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Weidinger, L., Uesato, J., Rauh, M., Griffin, C., Huang, P.-S., Mellor, J., Glaese, A., Cheng, M., Balle, B., and Kasirzadeh, A. (2022, January 21\u201324). Taxonomy of Risks Posed by Language Models. Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, Seoul, Republic of Korea.","DOI":"10.1145\/3531146.3533088"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Brown, H., Lee, K., Mireshghallah, F., Shokri, R., and Tram\u00e8r, F. (2022, January 21\u201324). What Does It Mean for a Language Model to Preserve Privacy?. Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, Seoul, Republic of Korea.","DOI":"10.1145\/3531146.3534642"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"16","DOI":"10.1016\/j.nbt.2023.02.001","article-title":"AI for Life: Trends in Artificial Intelligence for Biotechnology","volume":"74","author":"Holzinger","year":"2023","journal-title":"New Biotechnol."},{"key":"ref_22","unstructured":"Sharma, G., and Thakur, A. (2024, March 08). ChatGPT in Drug Discovery. Available online: https:\/\/chemrxiv.org\/engage\/chemrxiv\/article-details\/63d56c13ae221ab9b240932f."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"1098","DOI":"10.1111\/jerd.13046","article-title":"Implications of Large Language Models Such as ChatGPT for Dental Medicine","volume":"35","author":"Eggmann","year":"2023","journal-title":"J. Esthet. Restor. Dent."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., and Fritz, M. (2023). Not What You\u2019ve Signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. arXiv.","DOI":"10.1145\/3605764.3623985"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Harrer, S. (2023). Attention Is Not All You Need: The Complicated Case of Ethically Using Large Language Models in Healthcare and Medicine. eBioMedicine, 90.","DOI":"10.1016\/j.ebiom.2023.104512"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Li, J., Dada, A., Puladi, B., Kleesiek, J., and Egger, J. (2024). ChatGPT in Healthcare: A Taxonomy and Systematic Review. Comput. Methods Programs Biomed., 245.","DOI":"10.1016\/j.cmpb.2024.108013"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"120","DOI":"10.1038\/s41746-023-00873-0","article-title":"The Imperative for Regulatory Oversight of Large Language Models (or Generative AI) in Healthcare","volume":"6","author":"Topol","year":"2023","journal-title":"NPJ Digit. Med."},{"key":"ref_28","unstructured":"Derner, E., Batisti\u010d, K., Zah\u00e1lka, J., and Babu\u0161ka, R. (2023). A Security Risk Taxonomy for Large Language Models. arXiv."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"He, K., Mao, R., Lin, Q., Ruan, Y., Lan, X., Feng, M., and Cambria, E. (2023). A Survey of Large Language Models for Healthcare: From Data, Technology, and Applications to Accountability and Ethics. arXiv.","DOI":"10.2139\/ssrn.4809363"},{"key":"ref_30","unstructured":"Chen, J., Guo, H., Yi, K., Li, B., and Elhoseiny, M. (2021). VisualGPT: Data-Efficient Image Captioning by Balancing Visual Input and Linguistic Knowledge from Pretraining. CoRR, abs\/2102.10407."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Wang, S., Zhao, Z., Ouyang, X., Wang, Q., and Shen, D. (2023). ChatCAD: Interactive Computer-Aided Diagnosis on Medical Image Using Large Language Models. arXiv.","DOI":"10.1038\/s44172-024-00271-8"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Li, C., Zhang, Y., Weng, Y., Wang, B., and Li, Z. (2023). Natural Language Processing Applications for Computer-Aided Diagnosis in Oncology. Diagnostics, 13.","DOI":"10.3390\/diagnostics13020286"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"8839524","DOI":"10.1155\/2020\/8839524","article-title":"Text Messaging-Based Medical Diagnosis Using Natural Language Processing and Fuzzy Logic","volume":"2020","author":"Omoregbe","year":"2020","journal-title":"J. Healthc. Eng."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"e230163","DOI":"10.1148\/radiol.230163","article-title":"ChatGPT and Other Large Language Models Are Double-Edged Swords","volume":"307","author":"Shen","year":"2023","journal-title":"Radiology"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"e17828","DOI":"10.2196\/17828","article-title":"Perceptions and Opinions of Patients About Mental Health Chatbots: Scoping Review","volume":"23","author":"Alajlani","year":"2021","journal-title":"J. Med. Internet Res."},{"key":"ref_36","unstructured":"Ji, S., Zhang, T., Ansari, L., Fu, J., Tiwari, P., and Cambria, E. (2021). MentalBERT: Publicly Available Pretrained Language Models for Mental Healthcare. arXiv."},{"key":"ref_37","unstructured":"Carlini, N., Liu, C., Erlingsson, \u00da., Kos, J., and Song, D. (2019, January 14\u201316). The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networks. Proceedings of the 28th USENIX Security Symposium, Santa Clara, CA, USA."},{"key":"ref_38","unstructured":"Nasr, M., Shokri, R., and Houmansadr, A. (2019, January 19\u201323). Comprehensive Privacy Analysis of Deep Learning: Stand-Alone and Federated Learning under Passive and Active White-Box Inference Attacks. Proceedings of the 2019 IEEE Symposium on Security and Privacy, San Francisco, CA, USA."},{"key":"ref_39","unstructured":"Shokri, R., Stronati, M., and Shmatikov, V. (2016). Membership Inference Attacks against Machine Learning Models. CoRR, abs\/1610.05820."},{"key":"ref_40","unstructured":"Jagannatha, A., Rawat, B.P.S., and Yu, H. (2021). Membership Inference Attack Susceptibility of Clinical Language Models. CoRR, abs\/2104.08305."},{"key":"ref_41","doi-asserted-by":"crossref","first-page":"10207","DOI":"10.1109\/ACCESS.2023.3239668","article-title":"Membership Inference Attacks With Token-Level Deduplication on Korean Language Models","volume":"11","author":"Oh","year":"2023","journal-title":"IEEE Access"},{"key":"ref_42","unstructured":"Zhang, R., Hidano, S., and Koushanfar, F. (2022). Text Revealer: Private Text Reconstruction via Model Inversion Attacks against Transformers. arXiv."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"991","DOI":"10.1109\/TIFS.2022.3233190","article-title":"Label-Only Model Inversion Attacks: Attack With the Least Information","volume":"18","author":"Zhu","year":"2023","journal-title":"IEEE Trans. Inf. Forensics Secur."},{"key":"ref_44","unstructured":"Guo, S., Xie, C., Li, J., Lyu, L., and Zhang, T. (2022). Threats to Pre-Trained Language Models: Survey and Taxonomy. arXiv."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Li, S., Liu, H., Dong, T., Zhao, B.Z.H., Xue, M., Zhu, H., and Lu, J. (2021, January 15\u201319). Hidden Backdoors in Human-Centric Language Models. Proceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security, Virtual.","DOI":"10.1145\/3460120.3484576"},{"key":"ref_46","unstructured":"Yan, J., Yadav, V., LI, S., Chen, L., Tang, Z., Wang, H., Srinivasan, V., Ren, X., and Jin, H. (2024, January 16\u201321). Backdooring Instruction-Tuned Large Language Models with Virtual Prompt Injection. Proceedings of the 4th American Chapter of the Association for Computational Linguistics, Mexico City, Mexico."},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Lapid, R., Langberg, R., and Sipper, M. (2023). Open Sesame! Universal Black Box Jailbreaking of Large Language Models. arXiv.","DOI":"10.3390\/app14167150"},{"key":"ref_48","unstructured":"Shen, X., Chen, Z.J., Backes, M., Shen, Y., and Zhang, Y. (2023). \u201cDo Anything Now\u201d: Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models. arXiv."},{"key":"ref_49","unstructured":"Roman Samoilenko (2023, March 29). New Prompt Injection Attack on ChatGPT Web Version. Markdown Images Can Steal Your Chat Data Web Page. Available online: https:\/\/systemweakness.com\/new-prompt-injection-attack-on-chatgpt-web-version-ef717492c5c2."},{"key":"ref_50","doi-asserted-by":"crossref","unstructured":"Heidenreich, H.S., and Williams, J.R. (2021, January 19\u201321). The Earth Is Flat and the Sun Is Not a Star: The Susceptibility of GPT-2 to Universal Adversarial Triggers. Proceedings of the 2021 AAAI\/ACM Conference on AI, Ethics, and Society, Virtual.","DOI":"10.1145\/3461702.3462578"},{"key":"ref_51","first-page":"e39305","article-title":"Embracing Large Language Models for Medical Applications: Opportunities and Challenges","volume":"15","author":"Karabacak","year":"2023","journal-title":"Cureus"},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"1119","DOI":"10.1007\/s10439-023-03329-4","article-title":"ChatGPT and Clinical Decision Support: Scope, Application, and Limitations","volume":"52","author":"Ferdush","year":"2024","journal-title":"Ann. Biomed. Eng."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Nazi, Z.A., and Peng, W. (2024). Large Language Models in Healthcare and Medical Domain: A Review. Informatics, 11.","DOI":"10.3390\/informatics11030057"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Dave, T., Athaluri, S.A., and Singh, S. (2023). ChatGPT in Medicine: An Overview of Its Applications, Advantages, Limitations, Future Prospects, and Ethical Considerations. Front. Artif. Intell., 6.","DOI":"10.3389\/frai.2023.1169595"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Hossain, E., Rana, R., Higgins, N., Soar, J., Barua, P.D., Pisani, A.R., and Turner, K. (2023). Natural Language Processing in Electronic Health Records in Relation to Healthcare Decision-Making: A Systematic Review. Comput. Biol. Med., 155.","DOI":"10.1016\/j.compbiomed.2023.106649"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"20552076231186520","DOI":"10.1177\/20552076231186520","article-title":"Artificial Intelligence in Healthcare: Complementing, Not Replacing, Doctors and Healthcare Providers","volume":"9","author":"Sezgin","year":"2023","journal-title":"Digit. Health"},{"key":"ref_57","unstructured":"Zhao, W.X., Zhou, K., Li, J., Tang, T., Wang, X., Hou, Y., Min, Y., Zhang, B., Zhang, J., and Dong, Z. (2023). A Survey of Large Language Models. arXiv."},{"key":"ref_58","doi-asserted-by":"crossref","first-page":"456","DOI":"10.1177\/0706743719828977","article-title":"Chatbots and Conversational Agents in Mental Health: A Review of the Psychiatric Landscape","volume":"64","author":"Vaidyam","year":"2019","journal-title":"Can. J. Psychiatry"},{"key":"ref_59","doi-asserted-by":"crossref","first-page":"102642","DOI":"10.1016\/j.ijinfomgt.2023.102642","article-title":"Opinion Paper: \u201cSo What If ChatGPT Wrote It?\u201d Multidisciplinary Perspectives on Opportunities, Challenges and Implications of Generative Conversational AI for Research, Practice and Policy","volume":"71","author":"Dwivedi","year":"2023","journal-title":"Int. J. Inf. Manag."},{"key":"ref_60","doi-asserted-by":"crossref","unstructured":"Agbavor, F., and Liang, H. (2022). Predicting Dementia from Spontaneous Speech Using Large Language Models. PLoS Digit. Health, 1.","DOI":"10.1371\/journal.pdig.0000168"},{"key":"ref_61","unstructured":"Wong, C., Zhang, S., Gu, Y., Moung, C., Abel, J., Usuyama, N., Weerasinghe, R., Piening, B., Naumann, T., and Bifulco, C. (2023, January 11\u201312). Scaling Clinical Trial Matching Using Large Language Models: A Case Study in Oncology. Proceedings of the 8th Machine Learning for Healthcare Conference, New York, NY, USA."},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Hirosawa, T., Harada, Y., Yokose, M., Sakamoto, T., Kawamura, R., and Shimizu, T. (2023). Diagnostic Accuracy of Differential-Diagnosis Lists Generated by Generative Pretrained Transformer 3 Chatbot for Clinical Vignettes with Common Chief Complaints: A Pilot Study. Int. J. Environ. Res. Public Health, 20.","DOI":"10.3390\/ijerph20043378"},{"key":"ref_63","first-page":"44","article-title":"A Systematic Review of Natural Language Processing in Healthcare","volume":"8","author":"Olaronke","year":"2015","journal-title":"Int. J. Inf. Technol. Comput. Sci."}],"container-title":["Big Data and Cognitive Computing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-2289\/8\/11\/161\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:34:14Z","timestamp":1760114054000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-2289\/8\/11\/161"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,11,18]]},"references-count":63,"journal-issue":{"issue":"11","published-online":{"date-parts":[[2024,11]]}},"alternative-id":["bdcc8110161"],"URL":"https:\/\/doi.org\/10.3390\/bdcc8110161","relation":{},"ISSN":["2504-2289"],"issn-type":[{"value":"2504-2289","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,11,18]]}}}