{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,29]],"date-time":"2026-03-29T07:01:27Z","timestamp":1774767687820,"version":"3.50.1"},"reference-count":32,"publisher":"MDPI AG","issue":"8","license":[{"start":{"date-parts":[[2025,8,16]],"date-time":"2025-08-16T00:00:00Z","timestamp":1755302400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Computers"],"abstract":"<jats:p>Large language models (LLMs) now match or exceed human performance on many open-ended language tasks, yet they continue to produce fluent but incorrect statements, which is a failure mode widely referred to as hallucination. In low-stakes settings this may be tolerable; in regulated or safety-critical domains such as financial services, compliance review, and client decision support, it is not. Motivated by these realities, we develop an integrated mitigation framework that layers complementary controls rather than relying on any single technique. The framework combines structured prompt design, retrieval-augmented generation (RAG) with verifiable evidence sources, and targeted fine-tuning aligned with domain truth constraints. Our interest in this problem is practical. Individual mitigation techniques have matured quickly, yet teams deploying LLMs in production routinely report difficulty stitching them together in a coherent, maintainable pipeline. Decisions about when to ground a response in retrieved data, when to escalate uncertainty, how to capture provenance, and how to evaluate fidelity are often made ad hoc. Drawing on experience from financial technology implementations, where even rare hallucinations can carry material cost, regulatory exposure, or loss of customer trust, we aim to provide clearer guidance in the form of an easy-to-follow tutorial. This paper makes four contributions. First, we introduce a three-layer reference architecture that organizes mitigation activities across input governance, evidence-grounded generation, and post-response verification. Second, we describe a lightweight supervisory agent that manages uncertainty signals and triggers escalation (to humans, alternate models, or constrained workflows) when confidence falls below policy thresholds. Third, we analyze common but under-addressed security surfaces relevant to hallucination mitigation, including prompt injection, retrieval poisoning, and policy evasion attacks. Finally, we outline an implementation playbook for production deployment, including evaluation metrics, operational trade-offs, and lessons learned from early financial-services pilots.<\/jats:p>","DOI":"10.3390\/computers14080332","type":"journal-article","created":{"date-parts":[[2025,8,18]],"date-time":"2025-08-18T13:28:22Z","timestamp":1755523702000},"page":"332","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["Multi-Layered Framework for LLM Hallucination Mitigation in High-Stakes Applications: A Tutorial"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1091-970X","authenticated-orcid":false,"given":"Sachin","family":"Hiriyanna","sequence":"first","affiliation":[{"name":"Navan Inc., Palo Alto, CA 94306, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3202-1127","authenticated-orcid":false,"given":"Wenbing","family":"Zhao","sequence":"additional","affiliation":[{"name":"Department of Electrical and Computer Engineering, Cleveland State University, Cleveland, OH 44115, USA"}]}],"member":"1968","published-online":{"date-parts":[[2025,8,16]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3641289","article-title":"A survey on evaluation of large language models","volume":"15","author":"Chang","year":"2024","journal-title":"ACM Trans. Intell. Syst. Technol."},{"key":"ref_2","first-page":"43","article-title":"Unlocking the Potential: A Comprehensive Systematic Review of ChatGPT in Natural Language Processing Tasks","volume":"141","author":"Alomari","year":"2024","journal-title":"Comput. Model. Eng. Sci."},{"key":"ref_3","first-page":"2061","article-title":"Exploring the Latest Applications of OpenAI and ChatGPT: An In-Depth Survey","volume":"138","author":"Zhang","year":"2024","journal-title":"Comput. Model. Eng. Sci."},{"key":"ref_4","first-page":"1","article-title":"A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions","volume":"43","author":"Huang","year":"2025","journal-title":"ACM Trans. Inf. Syst."},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Shuster, K., Poff, S., Chen, M., Kiela, D., and Weston, J. (2021). Retrieval Augmentation Reduces Hallucination in Conversation. arXiv.","DOI":"10.18653\/v1\/2021.findings-emnlp.320"},{"key":"ref_6","unstructured":"Bouamor, H., Pino, J., and Bali, K. (2023). WikiChat: Stopping the Hallucination of Large Language Model Chatbots by Few-Shot Grounding on Wikipedia. Findings of the Association for Computational Linguistics: EMNLP 2023, Association for Computational Linguistics."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Darwish, A.M., Rashed, E.A., and Khoriba, G. (2025). Mitigating LLM Hallucinations Using a Multi-Agent Framework. Information, 16.","DOI":"10.3390\/info16070517"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Yu, S., Kim, G., and Kang, S. (2025). Context and Layers in Harmony: A Unified Strategy for Mitigating LLM Hallucinations. Mathematics, 13.","DOI":"10.20944\/preprints202504.1749.v1"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Li, F. (2025, January 21\u201323). MH-PEFT: Mitigating Hallucinations in Large Vision-Language Models through the PEFT Method. Proceedings of the 2025 2nd International Conference on Generative Artificial Intelligence and Information Security, Hangzhou, China.","DOI":"10.1145\/3728725.3728747"},{"key":"ref_10","unstructured":"Guan, X., Liu, Y., Lin, H., Lu, Y., He, B., Han, X., and Sun, L. (2024, January 20\u201327). Mitigating large language model hallucinations via autonomous knowledge graph-based retrofitting. Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence, AAAI\u201924\/IAAI\u201924\/EAAI\u201924, Vancouver, BC, Canada."},{"key":"ref_11","unstructured":"Moroney, L. (2025, July 22). The Trust Dilemma: Overcoming LLM Hallucinations in Financial Services. Available online: https:\/\/blog.chain.link\/the-trust-dilemma\/."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Zhao, W. (2021). From Traditional Fault Tolerance to Blockchain, John Wiley & Sons.","DOI":"10.1002\/9781119682127"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zhao, W., Yang, S., and Luo, X. (2019, January 15\u201318). On Consensus in Public Blockchains. Proceedings of the 2019 International Conference on Blockchain Technology, Honolulu, HI, USA.","DOI":"10.1145\/3320154.3320162"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"3546","DOI":"10.1109\/TDSC.2022.3193092","article-title":"On Next proof of stake algorithm: A simulation study","volume":"20","author":"Zhao","year":"2022","journal-title":"IEEE Trans. Dependable Secur. Comput."},{"key":"ref_15","unstructured":"Chen, W., Yang, Y., Kachuee, M., and Fu, X.Y. (May, January 29). Developing a Reliable, Fast, General-Purpose Hallucination Detection and Mitigation Service. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 3: Industry Track), Albuquerque, New Mexico."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"231","DOI":"10.1007\/s40747-025-01833-9","article-title":"Reducing hallucinations of large language models via hierarchical semantic piece","volume":"11","author":"Liu","year":"2025","journal-title":"Complex Intell. Syst."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1145\/514183.514185","article-title":"Principled design of the modern Web architecture","volume":"2","author":"Fielding","year":"2002","journal-title":"ACM Trans. Internet Technol."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1017\/S0269888900008122","article-title":"Intelligent agents: Theory and practice","volume":"10","author":"Wooldridge","year":"1995","journal-title":"Knowl. Eng. Rev."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"51","DOI":"10.1016\/0004-3702(93)90034-9","article-title":"Agent-oriented programming","volume":"60","author":"Shoham","year":"1993","journal-title":"Artif. Intell."},{"key":"ref_20","unstructured":"Balaguer, A., Benara, V., de Freitas Cunha, R.L., de Estev\u00e3o Filho, M.R., Hendry, T., Holstein, D., Marsman, J., Mecklenburg, N., Malvar, S., and Nunes, L.O. (2024). RAG vs Fine-tuning: Pipelines, Tradeoffs, and a Case Study on Agriculture. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Deng, G., Liu, Y., Li, Y., Wang, K., Zhang, Y., Li, Z., Wang, H., Zhang, T., and Liu, Y. (March, January 26). MASTERKEY: Automated Jailbreaking of Large Language Model Chatbots. Proceedings of the 2024 Network and Distributed System Security Symposium, San Diego, CA, USA. NDSS 2024.","DOI":"10.14722\/ndss.2024.24188"},{"key":"ref_22","unstructured":"Liu, Y., Deng, G., Li, Y., Wang, K., Wang, Z., Wang, X., Zhang, T., Liu, Y., Wang, H., and Zheng, Y. (2024). Prompt Injection attack against LLM-integrated Applications. arXiv."},{"key":"ref_23","unstructured":"Zou, W., Geng, R., Wang, B., and Jia, J. (2024). PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation of Large Language Models. arXiv."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Marzoev, A., Ara\u00fajo, L.T., Schwarzkopf, M., Yagati, S., Kohler, E., Morris, R., Kaashoek, M.F., and Madden, S. (2019, January 13\u201315). Towards Multiverse Databases. Proceedings of the Workshop on Hot Topics in Operating Systems, HotOS \u201919, New York, NY, USA.","DOI":"10.1145\/3317550.3321425"},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Lin, S., Hilton, J., and Evans, O. (2021). Truthfulqa: Measuring how models mimic human falsehoods. arXiv.","DOI":"10.18653\/v1\/2022.acl-long.229"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Min, S., Krishna, K., Lyu, X., Lewis, M., Yih, W.t., Koh, P.W., Iyyer, M., Zettlemoyer, L., and Hajishirzi, H. (2023). Factscore: Fine-grained atomic evaluation of factual precision in long form text generation. arXiv.","DOI":"10.18653\/v1\/2023.emnlp-main.741"},{"key":"ref_27","unstructured":"Hu, T., and Zhou, X.H. (2024). Unveiling llm evaluation focused on metrics: Challenges and solutions. arXiv."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"106700","DOI":"10.1016\/j.infsof.2021.106700","article-title":"Challenges and solutions when adopting DevSecOps: A systematic review","volume":"141","author":"Rajapakse","year":"2022","journal-title":"Inf. Softw. Technol."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Hu, X. (2025). Dynamics of Adversarial Attacks on Large Language Model-Based Search Engines. arXiv.","DOI":"10.2139\/ssrn.5083138"},{"key":"ref_30","unstructured":"Zhong, P.Y., Chen, S., Wang, R., McCall, M., Titzer, B.L., Miller, H., and Gibbons, P.B. (2025). RTBAS: Defending LLM Agents Against Prompt Injection and Privacy Leakage. arXiv."},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Zhou, H., Lee, K.H., Zhan, Z., Chen, Y., Li, Z., Wang, Z., Haddadi, H., and Yilmaz, E. (2025). TrustRAG: Enhancing Robustness and Trustworthiness in Retrieval-Augmented Generation. arXiv.","DOI":"10.32388\/Z4DWHQ"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Cohen, J. (2013). Statistical Power Analysis for the Behavioral Sciences, Routledge.","DOI":"10.4324\/9780203771587"}],"container-title":["Computers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2073-431X\/14\/8\/332\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:29:02Z","timestamp":1760034542000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2073-431X\/14\/8\/332"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,16]]},"references-count":32,"journal-issue":{"issue":"8","published-online":{"date-parts":[[2025,8]]}},"alternative-id":["computers14080332"],"URL":"https:\/\/doi.org\/10.3390\/computers14080332","relation":{},"ISSN":["2073-431X"],"issn-type":[{"value":"2073-431X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,16]]}}}