{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T01:34:12Z","timestamp":1760060052813,"version":"build-2065373602"},"reference-count":47,"publisher":"MDPI AG","issue":"3","license":[{"start":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T00:00:00Z","timestamp":1753833600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["MAKE"],"abstract":"<jats:p>The increasing complexity and scale of electronic health records (EHRs) demand advanced tools for efficient data retrieval, summarization, and comparative analysis in clinical practice. MERA (Medical Electronic Records Assistant) is a Retrieval-Augmented Generation (RAG)-based AI system that addresses these needs by integrating domain-specific retrieval with large language models (LLMs) to deliver robust question answering, similarity search, and report summarization functionalities. MERA is designed to overcome key limitations of conventional LLMs in healthcare, such as hallucinations, outdated knowledge, and limited explainability. To ensure both privacy compliance and model robustness, we constructed a large synthetic dataset using state-of-the-art LLMs, including Mistral v0.3, Qwen 2.5, and Llama 3, and further validated MERA on de-identified real-world EHRs from the MIMIC-IV-Note dataset. Comprehensive evaluation demonstrates MERA\u2019s high accuracy in medical question answering (correctness: 0.91; relevance: 0.98; groundedness: 0.89; retrieval relevance: 0.92), strong summarization performance (ROUGE-1 F1-score: 0.70; Jaccard similarity: 0.73), and effective similarity search (METEOR: 0.7\u20131.0 across diagnoses), with consistent results on real EHRs. The similarity search module empowers clinicians to efficiently identify and compare analogous patient cases, supporting differential diagnosis and personalized treatment planning. By generating concise, contextually relevant, and explainable insights, MERA reduces clinician workload and enhances decision-making. To our knowledge, this is the first system to integrate clinical question answering, summarization, and similarity search within a unified RAG-based framework.<\/jats:p>","DOI":"10.3390\/make7030073","type":"journal-article","created":{"date-parts":[[2025,7,30]],"date-time":"2025-07-30T07:37:23Z","timestamp":1753861043000},"page":"73","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["MERA: Medical Electronic Records Assistant"],"prefix":"10.3390","volume":"7","author":[{"ORCID":"https:\/\/orcid.org\/0009-0002-9217-4173","authenticated-orcid":false,"given":"Ahmed","family":"Ibrahim","sequence":"first","affiliation":[{"name":"AI Innovation Lab, Weill Cornell Medicine\u2014Qatar, Doha P.O. Box 24144, Qatar"}]},{"given":"Abdullah","family":"Khalili","sequence":"additional","affiliation":[{"name":"AI Innovation Lab, Weill Cornell Medicine\u2014Qatar, Doha P.O. Box 24144, Qatar"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0815-7902","authenticated-orcid":false,"given":"Maryam","family":"Arabi","sequence":"additional","affiliation":[{"name":"AI Innovation Lab, Weill Cornell Medicine\u2014Qatar, Doha P.O. Box 24144, Qatar"}]},{"given":"Aamenah","family":"Sattar","sequence":"additional","affiliation":[{"name":"AI Innovation Lab, Weill Cornell Medicine\u2014Qatar, Doha P.O. Box 24144, Qatar"},{"name":"Department of Medicine, New Vision University, 0159 Tbilisi, Georgia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2967-3033","authenticated-orcid":false,"given":"Abdullah","family":"Hosseini","sequence":"additional","affiliation":[{"name":"AI Innovation Lab, Weill Cornell Medicine\u2014Qatar, Doha P.O. Box 24144, Qatar"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4145-5509","authenticated-orcid":false,"given":"Ahmed","family":"Serag","sequence":"additional","affiliation":[{"name":"AI Innovation Lab, Weill Cornell Medicine\u2014Qatar, Doha P.O. Box 24144, Qatar"}]}],"member":"1968","published-online":{"date-parts":[[2025,7,30]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"299","DOI":"10.1007\/s10462-024-10921-0","article-title":"Large Language Models in Medical and Healthcare Fields: Applications, Advances, and Challenges","volume":"57","author":"Wang","year":"2024","journal-title":"Artif. Intell. Rev."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Sallam, M. (2023). The utility of ChatGPT as an example of large language models in healthcare education, research and practice: Systematic review on the future perspectives and potential limitations. MedRxiv, medRxiv:2023.02.19.23286155.","DOI":"10.1101\/2023.02.19.23286155"},{"key":"ref_3","first-page":"100658","article-title":"D3: A Small Language Model for Drug-Drug Interaction prediction and comparison with Large Language Models","volume":"20","author":"Ibrahim","year":"2025","journal-title":"Mach. Learn. Appl."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Ali, H., Qadir, J., and Shah, Z. (2023, January 16\u201317). Chatgpt and large language models (llms) in healthcare: Opportunities and risks. Proceedings of the 2023 IEEE International Conference on Artificial Intelligence, Blockchain, and Internet of Things (AIBThings), Mount Pleasant, MI, USA.","DOI":"10.1109\/AIBThings58340.2023.10291020"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1145\/3571730","article-title":"Survey of Hallucination in Natural Language Generation","volume":"55","author":"Ji","year":"2023","journal-title":"ACM Comput. Surv."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"618","DOI":"10.1038\/s41591-024-03445-1","article-title":"Medical large language models are vulnerable to data-poisoning attacks","volume":"31","author":"Alber","year":"2025","journal-title":"Nat. Med."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"288","DOI":"10.1038\/s41746-024-01282-7","article-title":"Medical large language models are susceptible to targeted misinformation attacks","volume":"7","author":"Han","year":"2024","journal-title":"NPJ Digit. Med."},{"key":"ref_8","first-page":"9459","article-title":"Retrieval-augmented generation for knowledge-intensive nlp tasks","volume":"33","author":"Lewis","year":"2020","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"AIra2400380","DOI":"10.1056\/AIra2400380","article-title":"RAG in Health Care: A Novel Framework for Improving Communication and Decision-Making by Addressing LLM Limitations","volume":"2","author":"Ng","year":"2025","journal-title":"NEJM AI"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"86","DOI":"10.4236\/jcc.2024.1211006","article-title":"Improving Clinical Support Through Retrieval-Augmented Generation Powered Virtual Health Assistants","volume":"12","author":"Anandavally","year":"2024","journal-title":"J. Comput. Commun."},{"key":"ref_11","first-page":"e42634","article-title":"Harnessing the power of large language models (LLMs) for electronic health records (EHRs) optimization","volume":"15","author":"Nashwan","year":"2023","journal-title":"Cureus"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"e26772","DOI":"10.1016\/j.heliyon.2024.e26772","article-title":"Multimodal risk prediction with physiological signals, medical images and clinical notes","volume":"10","author":"Wang","year":"2024","journal-title":"Heliyon"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Ben Rabah, C., Sattar, A., Ibrahim, A., and Serag, A. (2025). A Multimodal Deep Learning Model for the Classification of Breast Cancer Subtypes. Diagnostics, 15.","DOI":"10.3390\/diagnostics15080995"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Ben Rabah, C., Petropoulos, I.N., Malik, R.A., and Serag, A. (2025). Vision transformers for automated detection of diabetic peripheral neuropathy in corneal confocal microscopy images. Front. Imaging, 4.","DOI":"10.3389\/fimag.2025.1542128"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"136","DOI":"10.1038\/s41746-020-00341-z","article-title":"Fusion of medical imaging and electronic health records using deep learning: A systematic review and implementation guidelines","volume":"3","author":"Huang","year":"2020","journal-title":"NPJ Digit. Med."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"589","DOI":"10.1001\/jamainternmed.2023.1838","article-title":"Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum","volume":"183","author":"Ayers","year":"2023","journal-title":"JAMA Intern. Med."},{"key":"ref_17","unstructured":"Ku, L.W., Martins, A., and Srikumar, V. (2024, January 11\u201316). Medical Dialogue System: A Survey of Categories, Methods, Evaluation and Challenges. Proceedings of the Findings of the Association for Computational Linguistics: ACL 2024, Bangkok, Thailand."},{"key":"ref_18","unstructured":"Wu, S., Shabestari, B., and Xing, L. (2024, January 13\u201317). Optimizing ICU Readmission Prediction: A Comparative Evaluation of AI Tools. Proceedings of the International Workshop on Applications of Medical AI, Amsterdam, The Netherlands."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Zhang, W., and Zhang, J. (2025). Hallucination Mitigation for Retrieval-Augmented Large Language Models: A Review. Mathematics, 13.","DOI":"10.3390\/math13050856"},{"key":"ref_20","unstructured":"Yang, Y., Davani, A., Sil, A., and Kumar, A. (2024, January 16\u201321). Reducing hallucination in structured outputs via Retrieval-Augmented Generation. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 6: Industry Track), Mexico City, Mexico."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"1134","DOI":"10.1038\/s41591-024-02855-5","article-title":"Adapted large language models can outperform medical experts in clinical text summarization","volume":"30","author":"Blankemeier","year":"2024","journal-title":"Nat. Med."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"470","DOI":"10.1093\/jamia\/ocae312","article-title":"A dataset and benchmark for hospital course summarization with adapted large language models","volume":"32","author":"Aali","year":"2024","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_23","unstructured":"Nowaczyk, S., Biecek, P., Chung, N.C., Vallati, M., Skruch, P., Jaworek-Korjakowska, J., Parkinson, S., Nikitas, A., Atzm\u00fcller, M., and Kliegr, T. (October, January 30). Chatgpt-healthprompt. harnessing the power of xai in prompt-based healthcare decision support using chatgpt. Proceedings of the European Conference on Artificial Intelligence, Krak\u00f3w, Poland."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"2355","DOI":"10.3390\/make6040116","article-title":"Systematic Analysis of Retrieval-Augmented Generation-Based LLMs for Medical Chatbot Applications","volume":"6","author":"Bora","year":"2024","journal-title":"Mach. Learn. Knowl. Extr."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Quidwai, M.A., and Lagana, A. (2024). A RAG Chatbot for Precision Medicine of Multiple Myeloma. MedRxiv, medRxiv:2024.03.14.24304293.","DOI":"10.1101\/2024.03.14.24304293"},{"key":"ref_26","unstructured":"Demner-Fushman, D., Ananiadou, S., Thompson, P., and Ondov, B. (2024, January 20\u201325). Building Certified Medical Chatbots: Overcoming Unstructured Data Limitations with Modular RAG. Proceedings of the First Workshop on Patient-Oriented Language Processing (CL4Health) @ LREC-COLING 2024, Torino, Italia. Available online: https:\/\/aclanthology.org\/2024.cl4health-1.15\/."},{"key":"ref_27","first-page":"296","article-title":"Integrating Chatbot Functionality in a Patient Summary Based Healthcare System","volume":"316","author":"Vasili","year":"2024","journal-title":"Stud. Health Technol. Inform."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Neupane, S., Tripathi, H., Mitra, S., Bozorgzad, S., Mittal, S., Rahimi, S., and Amirlatifi, A. (2024, January 15\u201318). CLINICSUM: Utilizing Language Models for Generating Clinical Summaries from Patient-Doctor Conversations. Proceedings of the 2024 IEEE International Conference on Big Data (BigData), Washington, DC, USA.","DOI":"10.1109\/BigData62323.2024.10825266"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"313","DOI":"10.1007\/s41666-024-00159-4","article-title":"Clinical information retrieval: A literature review","volume":"8","author":"Sivarajkumar","year":"2024","journal-title":"J. Healthc. Inform. Res."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"209278","DOI":"10.1109\/ACCESS.2020.3037710","article-title":"A novel deep similarity learning approach to electronic health records data","volume":"8","author":"Gupta","year":"2020","journal-title":"IEEE Access"},{"key":"ref_31","unstructured":"Nalela, P. (2025). Leveraging Generative AI Through Prompt Engineering and Rigorous Validation to Create Comprehensive Synthetic Datasets for AI Training in Healthcare. arXiv."},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Hosseini, A., and Serag, A. (2025). Is synthetic data generation effective in maintaining clinical biomarkers? Investigating diffusion models across diverse imaging modalities. Front. Artif. Intell., 7.","DOI":"10.3389\/frai.2024.1454441"},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"59074","DOI":"10.1109\/ACCESS.2025.3555619","article-title":"Self-Supervised Learning Powered by Synthetic Data From Diffusion Models: Application to X-Ray Images","volume":"13","author":"Hosseini","year":"2025","journal-title":"IEEE Access"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Jadon, A., and Kumar, S. (2023, January 25\u201327). Leveraging generative ai models for synthetic data generation in healthcare: Balancing research and privacy. Proceedings of the 2023 International Conference on Smart Applications, Communications and Networking (SmartNets), Istanbul, Turkey.","DOI":"10.1109\/SmartNets58706.2023.10215825"},{"key":"ref_35","unstructured":"Jiang, A.Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D.S., Casas, D.D.L., Bressand, F., Lengyel, G., Lample, G., and Saulnier, L. (2023). Mistral 7B. arXiv."},{"key":"ref_36","unstructured":"Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., and Wei, H. (2025). Qwen2.5 Technical Report. arXiv."},{"key":"ref_37","unstructured":"Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Yang, A., and Fan, A. (2024). The llama 3 herd of models. arXiv."},{"key":"ref_38","unstructured":"D\u00e9jean, H., Clinchant, S., and Formal, T. (2024). A thorough comparison of cross-encoders and llms for reranking splade. arXiv."},{"key":"ref_39","doi-asserted-by":"crossref","unstructured":"Pradeep, R., Liu, Y., Zhang, X., Li, Y., Yates, A., and Lin, J. (2022, January 10\u201314). Squeezing Water from a Stone: A Bag of Tricks for Further Improving Cross-Encoder Effectiveness for Reranking. Proceedings of the European Conference on Information Retrieval, Stavanger, Norway.","DOI":"10.1007\/978-3-030-99736-6_44"},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Shool, S., Adimi, S., Saboori Amleshi, R., Bitaraf, E., Golpira, R., and Tara, M. (2025). A systematic review of large language model (LLM) evaluations in clinical medicine. BMC Med. Inform. Decis. Mak., 25.","DOI":"10.1186\/s12911-025-02954-4"},{"key":"ref_41","unstructured":"Hurst, A., Lerer, A., Goucher, A.P., Perelman, A., Ramesh, A., Clark, A., Ostrow, A., Welihinda, A., Hayes, A., and Radford, A. (2024). GPT-4o System Card. arXiv."},{"key":"ref_42","unstructured":"Lin, C.Y. (2004, January 25\u201326). Rouge: A package for automatic evaluation of summaries. Proceedings of the Text Summarization Branches Out, Barcelona, Spain."},{"key":"ref_43","unstructured":"Niwattanakul, S., Singthongchai, J., Naenudorn, E., and Wanapu, S. (2013, January 13\u201315). Using of Jaccard coefficient for keywords similarity. Proceedings of the International Multiconference Of Engineers and Computer Scientists, Hong Kong, China."},{"key":"ref_44","unstructured":"Goldstein, J., Lavie, A., Lin, C.Y., and Voss, C. (2005, January 29). METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments. Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and\/or Summarization, Ann Arbor, MI, USA. Available online: https:\/\/aclanthology.org\/W05-0909\/."},{"key":"ref_45","unstructured":"Johnson, A.E., Pollard, T.J., Horng, S., Celi, L.A., and Mark, R. (2025, July 01). MIMIC-IV-Note: Deidentified Free-Text Clinical Notes (Version 2.2). [Dataset] PhysioNet. RRID:SCR_007345. Available online: https:\/\/physionet.org\/content\/mimic-iv-note\/2.2\/."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"20552076251337177","DOI":"10.1177\/20552076251337177","article-title":"Enhancing medical AI with retrieval-augmented generation: A mini narrative review","volume":"11","author":"Gargari","year":"2025","journal-title":"Digit. Health"},{"key":"ref_47","unstructured":"Pu, X., Gao, M., and Wan, X. (2023). Summarization is (almost) dead. arXiv."}],"container-title":["Machine Learning and Knowledge Extraction"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2504-4990\/7\/3\/73\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T18:18:36Z","timestamp":1760033916000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2504-4990\/7\/3\/73"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,30]]},"references-count":47,"journal-issue":{"issue":"3","published-online":{"date-parts":[[2025,9]]}},"alternative-id":["make7030073"],"URL":"https:\/\/doi.org\/10.3390\/make7030073","relation":{},"ISSN":["2504-4990"],"issn-type":[{"type":"electronic","value":"2504-4990"}],"subject":[],"published":{"date-parts":[[2025,7,30]]}}}