{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,11]],"date-time":"2026-04-11T01:48:34Z","timestamp":1775872114657,"version":"3.50.1"},"reference-count":50,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,8,24]],"date-time":"2025-08-24T00:00:00Z","timestamp":1755993600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,8,24]],"date-time":"2025-08-24T00:00:00Z","timestamp":1755993600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Many rare genetic diseases have recognizable facial phenotypes that serve as diagnostic clues. While Large Language Models (LLMs) have shown potential in healthcare, their application to rare genetic diseases still faces challenges like hallucination and limited domain knowledge. To address these challenges, Retrieval-Augmented Generation (RAG) is an effective method, while Knowledge Graphs (KGs) provide more accurate and reliable information. In this paper, we constructed a Facial Phenotype Knowledge Graph (FPKG) including 6143 nodes and 19,282 relations and incorporate RAG to alleviate the hallucination of LLMs and enhance their ability to answer rare genetic disease questions. We evaluated eight LLMs across four tasks: domain-specific QA, diagnostic tests, consistency evaluation, and temperature analysis. The results showed that our approach improves both diagnostic accuracy and response consistency. Notably, RAG reduces temperature-induced variability by 53.94%. This study demonstrates that LLMs can effectively incorporate domain-specific KGs to enhance accuracy, and consistency, thereby improving diagnostic decision-making.<\/jats:p>","DOI":"10.1038\/s41746-025-01955-x","type":"journal-article","created":{"date-parts":[[2025,8,24]],"date-time":"2025-08-24T01:42:46Z","timestamp":1755999766000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":6,"title":["Graph retrieval augmented large language models for facial phenotype associated rare genetic disease"],"prefix":"10.1038","volume":"8","author":[{"given":"Jie","family":"Song","sequence":"first","affiliation":[]},{"given":"Zhichuan","family":"Xu","sequence":"additional","affiliation":[]},{"given":"Mengqiao","family":"He","sequence":"additional","affiliation":[]},{"given":"Jinhua","family":"Feng","sequence":"additional","affiliation":[]},{"given":"Bairong","family":"Shen","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2025,8,24]]},"reference":[{"key":"1955_CR1","first-page":"1403","volume":"9","author":"Z-L Li","year":"2016","unstructured":"Li, Z.-L. et al. FGFR2 mutation in a Chinese family with unusual Crouzon syndrome. Int. J. Ophthalmol. 9, 1403 (2016).","journal-title":"Int. J. Ophthalmol."},{"key":"1955_CR2","doi-asserted-by":"publisher","first-page":"60","DOI":"10.1038\/s41591-018-0279-0","volume":"25","author":"Y Gurovich","year":"2019","unstructured":"Gurovich, Y. et al. Identifying facial phenotypes of genetic disorders using deep learning. Nat. Med. 25, 60\u201364 (2019).","journal-title":"Nat. Med."},{"key":"1955_CR3","doi-asserted-by":"publisher","first-page":"349","DOI":"10.1038\/s41588-021-01010-x","volume":"54","author":"T-C Hsieh","year":"2022","unstructured":"Hsieh, T.-C. et al. GestaltMatcher facilitates rare disease matching using facial phenotype descriptors. Nat. Genet. 54, 349\u2013357 (2022).","journal-title":"Nat. Genet."},{"key":"1955_CR4","doi-asserted-by":"publisher","first-page":"1598","DOI":"10.1038\/s41588-023-01469-w","volume":"55","author":"AJ Dingemans","year":"2023","unstructured":"Dingemans, A. J. et al. PhenoScore quantifies phenotypic variation for rare genetic diseases by combining facial analysis with other clinical features using a machine-learning framework. Nat. Genet. 55, 1598\u20131607 (2023).","journal-title":"Nat. Genet."},{"key":"1955_CR5","unstructured":"Vaswani, A. et al. Attention is all you need. Advances in neural information processing systems 30 (2017)."},{"key":"1955_CR6","unstructured":"Achiam, J. et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)."},{"key":"1955_CR7","unstructured":"Anthropic. Introducing the next generation of Claude, <https:\/\/www.anthropic.com\/news\/claude-3-family> (2024)."},{"key":"1955_CR8","unstructured":"Team, G. et al. Gemini: A family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 (2023)."},{"key":"1955_CR9","unstructured":"Touvron, H. et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288 (2023)."},{"key":"1955_CR10","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01010-1","volume":"7","author":"T Savage","year":"2024","unstructured":"Savage, T., Nayak, A., Gallo, R., Rangan, E. & Chen, J. H. Diagnostic reasoning prompts reveal the potential for large language model interpretability in medicine. NPJ Digital Med. 7, 20 (2024).","journal-title":"NPJ Digital Med."},{"key":"1955_CR11","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01011-0","volume":"7","author":"BM Spiegel","year":"2024","unstructured":"Spiegel, B. M. et al. Feasibility of combining spatial computing and AI for mental health support in anxiety and depression. NPJ Digital Med. 7, 22 (2024).","journal-title":"NPJ Digital Med."},{"key":"1955_CR12","doi-asserted-by":"publisher","first-page":"2306724","DOI":"10.1002\/advs.202306724","volume":"11","author":"RK Luu","year":"2024","unstructured":"Luu, R. K. & Buehler, M. J. BioinspiredLLM: Conversational Large Language Model for the Mechanics of Biological and Bio-Inspired Materials. Adv. Sci. 11, 2306724 (2024).","journal-title":"Adv. Sci."},{"key":"1955_CR13","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-023-00989-3","volume":"7","author":"H Wang","year":"2024","unstructured":"Wang, H., Gao, C., Dantona, C., Hull, B. & Sun, J. DRG-LLaMA: Tuning LLaMA model to predict diagnosis-related group for hospitalized patients. npj Digital Med. 7, 16 (2024).","journal-title":"npj Digital Med."},{"key":"1955_CR14","doi-asserted-by":"publisher","first-page":"639","DOI":"10.1038\/s41433-023-02759-7","volume":"38","author":"E Waisberg","year":"2024","unstructured":"Waisberg, E., Ong, J., Masalkhi, M. & Lee, A. G. Large language model (LLM)-driven chatbots for neuro-ophthalmic medical education. Eye 38, 639\u2013641 (2024).","journal-title":"Eye"},{"key":"1955_CR15","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3571730","volume":"55","author":"Z Ji","year":"2023","unstructured":"Ji, Z. et al. Survey of hallucination in natural language generation. ACM Comput. Surv. 55, 1\u201338 (2023).","journal-title":"ACM Comput. Surv."},{"key":"1955_CR16","unstructured":"Kandpal, N., Deng, H., Roberts, A., Wallace, E. & Raffel, C. In International Conference on Machine Learning. 15696-15707 (PMLR)."},{"key":"1955_CR17","unstructured":"Chen, J., Lin, H., Han, X. & Sun, L. In Proceedings of the AAAI Conference on Artificial Intelligence. 17754-17762."},{"key":"1955_CR18","unstructured":"Gao, Y. et al. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997 (2023)."},{"key":"1955_CR19","unstructured":"Sanmartin, D. K. G.-R. A. G.: Bridging the Gap Between Knowledge and Creativity. arXiv preprint arXiv:2405.12035 (2024)."},{"key":"1955_CR20","unstructured":"Jiang, X. et al. Think and retrieval: A hypothesis knowledge graph enhanced medical large language models. arXiv preprint arXiv:2312.15883 (2023)."},{"key":"1955_CR21","unstructured":"Cavalleri, E. et al. SPIREX: Improving LLM-based relation extraction from RNA-focused scientific literature using graph machine learning. Proceedings of the VLDB Endowment. ISSN 2150, 8097"},{"key":"1955_CR22","doi-asserted-by":"publisher","first-page":"035083","DOI":"10.1088\/2632-2153\/ad7228","volume":"5","author":"MJ Buehler","year":"2024","unstructured":"Buehler, M. J. Accelerating scientific discovery with generative knowledge extraction, graph-based representation, and multimodal intelligent graph reasoning. Mach. Learn. Sci. Technol. 5, 035083 (2024).","journal-title":"Mach. Learn. Sci. Technol."},{"key":"1955_CR23","doi-asserted-by":"publisher","first-page":"241","DOI":"10.1021\/acsengineeringau.3c00058","volume":"4","author":"MJ Buehler","year":"2024","unstructured":"Buehler, M. J. Generative retrieval-augmented ontologic graph and multiagent strategies for interpretive large language model-based materials design. ACS Eng. Au 4, 241\u2013277 (2024).","journal-title":"ACS Eng. Au"},{"key":"1955_CR24","doi-asserted-by":"publisher","first-page":"D1207","DOI":"10.1093\/nar\/gkaa1043","volume":"49","author":"S K\u00f6hler","year":"2021","unstructured":"K\u00f6hler, S. et al. The human phenotype ontology in 2021. Nucleic acids Res. 49, D1207\u2013D1217 (2021).","journal-title":"Nucleic acids Res."},{"key":"1955_CR25","unstructured":"Zhang, T., Kishore, V., Wu, F., Weinberger, K. Q. & Artzi, Y. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675 (2019)."},{"key":"1955_CR26","unstructured":"Wang, X. et al. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171 (2022)."},{"key":"1955_CR27","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-023-00939-z","volume":"6","author":"JA Omiye","year":"2023","unstructured":"Omiye, J. A., Lester, J. C., Spichak, S., Rotemberg, V. & Daneshjou, R. Large language models propagate race-based medicine. NPJ Digital Med. 6, 195 (2023).","journal-title":"NPJ Digital Med."},{"key":"1955_CR28","unstructured":"Peeperkorn, M., Kouwenhoven, T., Brown, D. & Jordanous, A. Is temperature the creativity parameter of large language models? arXiv preprint arXiv:2405.00492 (2024)."},{"key":"1955_CR29","unstructured":"Zhu, Y. et al. In Proceedings of the AAAI Conference on Artificial Intelligence. 437-445."},{"key":"1955_CR30","doi-asserted-by":"publisher","first-page":"235","DOI":"10.1159\/000351127","volume":"4","author":"N Corsten-Janssen","year":"2013","unstructured":"Corsten-Janssen, N. et al. More clinical overlap between 22q11. 2 deletion syndrome and CHARGE syndrome than often anticipated. Mol. Syndromol. 4, 235\u2013245 (2013).","journal-title":"Mol. Syndromol."},{"key":"1955_CR31","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01029-4","volume":"7","author":"L Wang","year":"2024","unstructured":"Wang, L. et al. Prompt engineering in consistency and reliability with the evidence-based guideline for LLMs. npj Digital Med. 7, 41 (2024).","journal-title":"npj Digital Med."},{"key":"1955_CR32","doi-asserted-by":"crossref","unstructured":"Antaki, F. et al. Capabilities of GPT-4 in ophthalmology: An analysis of model entropy and progress towards human-level medical question answering. British Journal of Ophthalmology (2023).","DOI":"10.1136\/bjo-2023-324438"},{"key":"1955_CR33","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-01519-z","volume":"8","author":"YH Ke","year":"2025","unstructured":"Ke, Y. H. et al. Retrieval augmented generation for 10 large language models and its generalizability in assessing medical fitness. npj Digital Med. 8, 187 (2025).","journal-title":"npj Digital Med."},{"key":"1955_CR34","doi-asserted-by":"publisher","first-page":"1158","DOI":"10.1097\/HEP.0000000000000834","volume":"80","author":"J Ge","year":"2024","unstructured":"Ge, J. et al. Development of a liver disease\u2013specific large language model chat interface using retrieval-augmented generation. Hepatology 80, 1158\u20131168 (2024).","journal-title":"Hepatology"},{"key":"1955_CR35","doi-asserted-by":"publisher","first-page":"e58041","DOI":"10.2196\/58041","volume":"26","author":"D Wang","year":"2024","unstructured":"Wang, D. et al. Enhancement of the performance of large language models in diabetes education through retrieval-augmented generation: Comparative study. J. Med. Internet Res. 26, e58041 (2024).","journal-title":"J. Med. Internet Res."},{"key":"1955_CR36","doi-asserted-by":"publisher","first-page":"885","DOI":"10.1002\/ajmg.a.61124","volume":"179","author":"CR Ferreira","year":"2019","unstructured":"Ferreira, C. R. The burden of rare diseases. Am. J. Med. Genet. Part A 179, 885\u2013892 (2019).","journal-title":"Am. J. Med. Genet. Part A"},{"key":"1955_CR37","first-page":"677","volume":"42","author":"PA Baird","year":"1988","unstructured":"Baird, P. A., Anderson, T. W., Newcombe, H. B. & Lowry, R. Genetic disorders in children and young adults: a population study. Am. J. Hum. Genet. 42, 677 (1988).","journal-title":"Am. J. Hum. Genet."},{"key":"1955_CR38","doi-asserted-by":"publisher","first-page":"1884","DOI":"10.1093\/bioinformatics\/btab019","volume":"37","author":"L Luo","year":"2021","unstructured":"Luo, L. et al. PhenoTagger: A hybrid method for phenotype concept recognition using human phenotype ontology. Bioinformatics 37, 1884\u20131890 (2021).","journal-title":"Bioinformatics"},{"key":"1955_CR39","doi-asserted-by":"publisher","first-page":"W518","DOI":"10.1093\/nar\/gkt441","volume":"41","author":"C-H Wei","year":"2013","unstructured":"Wei, C.-H., Kao, H.-Y. & Lu, Z. PubTator: a web-based text mining tool for assisting biocuration. Nucleic acids Res. 41, W518\u2013W522 (2013).","journal-title":"Nucleic acids Res."},{"key":"1955_CR40","doi-asserted-by":"publisher","first-page":"4511","DOI":"10.1093\/bioinformatics\/btz385","volume":"35","author":"JP Buchmann","year":"2019","unstructured":"Buchmann, J. P. & Holmes, E. C. Entrezpy: A Python library to dynamically interact with the NCBI Entrez databases. Bioinformatics 35, 4511\u20134514 (2019).","journal-title":"Bioinformatics"},{"key":"1955_CR41","doi-asserted-by":"publisher","first-page":"1234","DOI":"10.1093\/bioinformatics\/btz682","volume":"36","author":"J Lee","year":"2020","unstructured":"Lee, J. et al. BioBERT: A pre-trained biomedical language representation model for biomedical text mining. Bioinformatics 36, 1234\u20131240 (2020).","journal-title":"Bioinformatics"},{"key":"1955_CR42","doi-asserted-by":"publisher","first-page":"cnaa007","DOI":"10.1093\/comnet\/cnaa007","volume":"8","author":"L Torres","year":"2020","unstructured":"Torres, L., Chan, K. S. & Eliassi-Rad, T. GLEE: Geometric Laplacian eigenmap embedding. J. Complex Netw. 8, cnaa007 (2020).","journal-title":"J. Complex Netw."},{"key":"1955_CR43","unstructured":"Lesmann, H. et al. GestaltMatcher Database-A global reference for facial phenotypic variability in rare human diseases. Res. Square, rs. 3. rs-4438861 (2024)."},{"key":"1955_CR44","doi-asserted-by":"publisher","first-page":"776","DOI":"10.1016\/j.gene.2015.11.006","volume":"576","author":"B Lee","year":"2016","unstructured":"Lee, B. et al. Revealing the function of a novel splice-site mutation of CHD7 in CHARGE syndrome. Gene 576, 776\u2013781 (2016).","journal-title":"Gene"},{"key":"1955_CR45","unstructured":"Perozzi, B., Al-Rfou, R. & Skiena, S. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. 701-710."},{"key":"1955_CR46","unstructured":"Grover, A. & Leskovec, J. In Proceedings of the 22nd ACM SIGKDD international conference on Knowledge discovery and data mining. 855-864."},{"key":"1955_CR47","doi-asserted-by":"crossref","unstructured":"Perozzi, B., Kulkarni, V., Chen, H. & Skiena, S. In Proceedings of the 2017 IEEE\/ACM International Conference on Advances in Social Networks Analysis and Mining 2017. 258-265.","DOI":"10.1145\/3110025.3110086"},{"key":"1955_CR48","unstructured":"Tang, L. & Liu, H. In Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining. 817-826."},{"key":"1955_CR49","doi-asserted-by":"crossref","unstructured":"Zhang, Z., Cui, P., Li, H., Wang, X. & Zhu, W. In 2018 IEEE international conference on data mining (ICDM). 787-796 (IEEE).","DOI":"10.1109\/ICDM.2018.00094"},{"key":"1955_CR50","unstructured":"Qiu, J. et al. In Proceedings of the eleventh ACM international conference on web search and data mining. 459-467."}],"updated-by":[{"DOI":"10.1038\/s41746-025-02017-y","type":"correction","label":"Correction","source":"publisher","updated":{"date-parts":[[2025,10,8]],"date-time":"2025-10-08T00:00:00Z","timestamp":1759881600000}}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-025-01955-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-025-01955-x","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-025-01955-x.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,9]],"date-time":"2025-10-09T05:29:44Z","timestamp":1759987784000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-025-01955-x"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,8,24]]},"references-count":50,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["1955"],"URL":"https:\/\/doi.org\/10.1038\/s41746-025-01955-x","relation":{"correction":[{"id-type":"doi","id":"10.1038\/s41746-025-02017-y","asserted-by":"object"}]},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,8,24]]},"assertion":[{"value":"27 May 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"15 August 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"24 August 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 October 2025","order":4,"name":"change_date","label":"Change Date","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"Correction","order":5,"name":"change_type","label":"Change Type","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"A Correction to this paper has been published:","order":6,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"https:\/\/doi.org\/10.1038\/s41746-025-02017-y","URL":"https:\/\/doi.org\/10.1038\/s41746-025-02017-y","order":7,"name":"change_details","label":"Change Details","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"543"}}