{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,27]],"date-time":"2026-10-27T08:19:23Z","timestamp":1793089163568,"version":"4.5.0"},"reference-count":38,"publisher":"American Society of Clinical Oncology (ASCO)","issue":"8","content-domain":{"domain":["ascopubs.org"],"crossmark-restriction":true},"short-container-title":["JCO Precis Oncol"],"published-print":{"date-parts":[[2024,10]]},"abstract":"<jats:sec>\n            <jats:title>PURPOSE<\/jats:title>\n            <jats:p>\n              Rapidly expanding medical literature challenges oncologists seeking targeted cancer therapies. General-purpose large language models (LLMs) lack domain-specific knowledge, limiting their clinical utility. This study introduces the LLM system Medical Evidence Retrieval and Data Integration for Tailored Healthcare (MEREDITH), designed to support treatment recommendations in precision oncology. Built on\n              <jats:italic>Google's Gemini Pro<\/jats:italic>\n              LLM, MEREDITH uses\n              <jats:italic>retrieval-augmented generation<\/jats:italic>\n              and\n              <jats:italic>chain of thought<\/jats:italic>\n              .\n            <\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>METHODS<\/jats:title>\n            <jats:p>\n              We evaluated MEREDITH on 10 publicly available fictional oncology cases with iterative feedback from a molecular tumor board (MTB) at a major German cancer center. Initially limited to\n              <jats:italic>PubMed<\/jats:italic>\n              -indexed literature (draft system), MEREDITH was enhanced to incorporate clinical studies on drug response within the specific tumor type, trial databases, drug approval status, and oncologic guidelines. The MTB provided a benchmark with manually curated treatment recommendations and assessed the clinical relevance of LLM-generated options (qualitative assessment). We measured semantic cosine similarity between LLM suggestions and clinician responses (quantitative assessment).\n            <\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>RESULTS<\/jats:title>\n            <jats:p>\n              MEREDITH identified a broader range of treatment options (median 4) compared with MTB experts (median 2). These options included therapies on the basis of preclinical data and combination treatments, expanding the treatment possibilities for consideration by the MTB. This broader approach was achieved by incorporating a curated medical data set that contextualized molecular targetability. Mirroring the approach MTB experts use to evaluate MTB cases improved the LLM's ability to generate relevant suggestions. This is supported by high concordance between LLM suggestions and expert recommendations (94.7% for the enhanced system) and a significant increase in semantic similarity from the draft to the enhanced system (from 0.71 to 0.76,\n              <jats:italic>P<\/jats:italic>\n              = .01).\n            <\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>CONCLUSION<\/jats:title>\n            <jats:p>Expert feedback and domain-specific data augment LLM performance. Future research should investigate responsible LLM integration into real-world clinical workflows.<\/jats:p>\n          <\/jats:sec>","DOI":"10.1200\/po-24-00478","type":"journal-article","created":{"date-parts":[[2024,10,30]],"date-time":"2024-10-30T19:59:09Z","timestamp":1730318349000},"update-policy":"https:\/\/doi.org\/10.1200\/crossmark","source":"Crossref","is-referenced-by-count":67,"title":["Expert-Guided Large Language Models for Clinical Decision Support in Precision Oncology"],"prefix":"10.1200","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0380-574X","authenticated-orcid":true,"given":"Jacqueline","family":"Lammert","sequence":"first","affiliation":[{"name":"Department of Gynecology and Center for Hereditary Breast and Ovarian Cancer, Technical University of Munich (TUM), School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital, Munich, Germany"},{"name":"Center for Personalized Medicine (ZPM), Technical University of Munich (TUM), School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital, Munich, Germany"},{"name":"German Cancer Consortium (DKTK), partner site Munich, a partnership between DKFZ and TUM University Hospital, Munich, Germany"},{"name":"EUropean Network for RAre CANcers (EURACAN) Initiative, partner site Munich, Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5366-7859","authenticated-orcid":true,"given":"Tobias","family":"Dreyer","sequence":"additional","affiliation":[{"name":"Department of Gynecology and Center for Hereditary Breast and Ovarian Cancer, Technical University of Munich (TUM), School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital, Munich, Germany"},{"name":"German Cancer Consortium (DKTK), partner site Munich, a partnership between DKFZ and TUM University Hospital, Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sonja","family":"Mathes","sequence":"additional","affiliation":[{"name":"Department of Dermatology and Allergy Biederstein, Technical University of Munich (TUM), School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital, Munich, Germany"},{"name":"Institute for History, Theory and Ethics of Medicine, University of Mainz Medical Center, Mainz, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Leonid","family":"Kuligin","sequence":"additional","affiliation":[{"name":"Google Cloud, Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kai J.","family":"Borm","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Technical University of Munich (TUM), School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital, Munich, Germany"},{"name":"Bavarian Center for Cancer Research (BZKF), Technical University of Munich partner site, Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0008-6596-128X","authenticated-orcid":true,"given":"Ulrich A.","family":"Schatz","sequence":"additional","affiliation":[{"name":"Department of Gynecology and Center for Hereditary Breast and Ovarian Cancer, Technical University of Munich (TUM), School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital, Munich, Germany"},{"name":"Center for Personalized Medicine (ZPM), Technical University of Munich (TUM), School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital, Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4237-2657","authenticated-orcid":true,"given":"Marion","family":"Kiechle","sequence":"additional","affiliation":[{"name":"Department of Gynecology and Center for Hereditary Breast and Ovarian Cancer, Technical University of Munich (TUM), School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital, Munich, Germany"},{"name":"Center for Personalized Medicine (ZPM), Technical University of Munich (TUM), School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital, Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alisa M.","family":"L\u00f6rsch","sequence":"additional","affiliation":[{"name":"Center for Personalized Medicine (ZPM), Technical University of Munich (TUM), School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital, Munich, Germany"},{"name":"Department of Medicine III, Technical University of Munich (TUM), School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital, Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0137-7929","authenticated-orcid":true,"given":"Johannes","family":"Jung","sequence":"additional","affiliation":[{"name":"Center for Personalized Medicine (ZPM), Technical University of Munich (TUM), School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital, Munich, Germany"},{"name":"Department of Medicine III, Technical University of Munich (TUM), School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital, Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0406-1401","authenticated-orcid":true,"given":"Sebastian","family":"Lange","sequence":"additional","affiliation":[{"name":"Center for Personalized Medicine (ZPM), Technical University of Munich (TUM), School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital, Munich, Germany"},{"name":"Department of Medicine II, Technical University of Munich (TUM), School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital, Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5977-5375","authenticated-orcid":true,"given":"Nicole","family":"Pfarr","sequence":"additional","affiliation":[{"name":"Center for Personalized Medicine (ZPM), Technical University of Munich (TUM), School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital, Munich, Germany"},{"name":"Institute of Pathology, TUM School of Medicine and Health, Technical University of Munich (TUM), Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0993-8869","authenticated-orcid":true,"given":"Anna","family":"Durner","sequence":"additional","affiliation":[{"name":"Center for Personalized Medicine (ZPM), Technical University of Munich (TUM), School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital, Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kristina","family":"Schwamborn","sequence":"additional","affiliation":[{"name":"Center for Personalized Medicine (ZPM), Technical University of Munich (TUM), School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital, Munich, Germany"},{"name":"Institute of Pathology, TUM School of Medicine and Health, Technical University of Munich (TUM), Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0253-9056","authenticated-orcid":true,"given":"Christof","family":"Winter","sequence":"additional","affiliation":[{"name":"Center for Personalized Medicine (ZPM), Technical University of Munich (TUM), School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital, Munich, Germany"},{"name":"German Cancer Consortium (DKTK), partner site Munich, a partnership between DKFZ and TUM University Hospital, Munich, Germany"},{"name":"Institute of Clinical Chemistry and Pathobiochemistry, Technical University of Munich (TUM), School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital, Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-6195-9276","authenticated-orcid":true,"given":"Dyke","family":"Ferber","sequence":"additional","affiliation":[{"name":"Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany"},{"name":"National Center for Tumor Diseases, Heidelberg University Hospital, Heidelberg, Germany"},{"name":"Department of Medical Oncology, Heidelberg University Hospital, Heidelberg, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3730-5348","authenticated-orcid":true,"given":"Jakob Nikolas","family":"Kather","sequence":"additional","affiliation":[{"name":"Else Kroener Fresenius Center for Digital Health, Technical University Dresden, Dresden, Germany"},{"name":"National Center for Tumor Diseases, Heidelberg University Hospital, Heidelberg, Germany"},{"name":"Department of Medicine I, University Hospital Dresden, Dresden, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Carolin","family":"Mogler","sequence":"additional","affiliation":[{"name":"Center for Personalized Medicine (ZPM), Technical University of Munich (TUM), School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital, Munich, Germany"},{"name":"German Cancer Consortium (DKTK), partner site Munich, a partnership between DKFZ and TUM University Hospital, Munich, Germany"},{"name":"Institute of Pathology, TUM School of Medicine and Health, Technical University of Munich (TUM), Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Anna L.","family":"Illert","sequence":"additional","affiliation":[{"name":"Center for Personalized Medicine (ZPM), Technical University of Munich (TUM), School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital, Munich, Germany"},{"name":"German Cancer Consortium (DKTK), partner site Munich, a partnership between DKFZ and TUM University Hospital, Munich, Germany"},{"name":"Bavarian Center for Cancer Research (BZKF), Technical University of Munich partner site, Munich, Germany"},{"name":"Department of Medicine III, Technical University of Munich (TUM), School of Medicine and Health, Klinikum rechts der Isar, TUM University Hospital, Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-7812-3698","authenticated-orcid":true,"given":"Maximilian","family":"Tschochohei","sequence":"additional","affiliation":[{"name":"Google Cloud, Munich, Germany"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"233","reference":[{"key":"e_1_3_9_2_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41587-022-01353-8"},{"key":"e_1_3_9_3_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41586-020-1969-6"},{"key":"e_1_3_9_4_2","doi-asserted-by":"publisher","DOI":"10.1186\/s13059-014-0438-7"},{"key":"e_1_3_9_5_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-023-02354-z"},{"key":"e_1_3_9_6_2","doi-asserted-by":"publisher","DOI":"10.1158\/2159-8290.CD-21-0126"},{"key":"e_1_3_9_7_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41571-023-00824-4"},{"key":"e_1_3_9_8_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ccell.2022.04.002"},{"key":"e_1_3_9_9_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41698-023-00458-w"},{"key":"e_1_3_9_10_2","doi-asserted-by":"publisher","DOI":"10.1038\/s43856-023-00370-1"},{"key":"e_1_3_9_11_2","doi-asserted-by":"publisher","DOI":"10.1001\/jamanetworkopen.2023.43689"},{"key":"e_1_3_9_12_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41591-023-02594-z"},{"key":"e_1_3_9_13_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41698-024-00517-w"},{"key":"e_1_3_9_14_2","first-page":"9459","volume-title":"Proceedings of the 34th International Conference on Neural Information Processing Systems","author":"Lewis P","year":"2020","unstructured":"Lewis P, Perez E, Piktus A, et al: Retrieval-augmented generation for knowledge-intensive NLP tasks, in Proceedings of the 34th International Conference on Neural Information Processing Systems. Red Hook, NY, Curran Associates, 2020. pp 9459-9474 (NIPS \u201920)"},{"key":"e_1_3_9_15_2","article-title":"Almanac: Retrieval-augmented language models for clinical medicine","author":"Zakka C","year":"2023","unstructured":"Zakka C, Chaurasia A, Shad R, et al: Almanac: Retrieval-augmented language models for clinical medicine. Res Sq, 2023. http:\/\/dx.doi.org\/10.21203\/rs.3.rs-2883198\/v1","journal-title":"Res Sq"},{"key":"e_1_3_9_16_2","article-title":"Comparison of treatment recommendations by molecular tumor boards worldwide","author":"Rieke DT","unstructured":"Rieke DT, Lamping M, Schuh M, et al: Comparison of treatment recommendations by molecular tumor boards worldwide. JCO Precis Oncol 10.1200\/PO.18.00098","journal-title":"JCO Precis Oncol"},{"key":"e_1_3_9_17_2","doi-asserted-by":"publisher","DOI":"10.1038\/gim.2015.30"},{"key":"e_1_3_9_18_2","doi-asserted-by":"publisher","DOI":"10.1145\/3641289"},{"key":"e_1_3_9_19_2","doi-asserted-by":"publisher","DOI":"10.1007\/s40192-024-00356-4"},{"key":"e_1_3_9_20_2","article-title":"Gemini: A family of highly capable multimodal models","author":"Gemini Team","year":"2023","unstructured":"Gemini Team; Anil R, Borgeaud S, Alayrac JB, et al: Gemini: A family of highly capable multimodal models. arXiv [cs.CL], 2023. http:\/\/arxiv.org\/abs\/2312.11805","journal-title":"arXiv [cs.CL]"},{"key":"e_1_3_9_21_2","unstructured":"Pymed [Internet]: PyPI. https:\/\/pypi.org\/project\/pymed\/"},{"key":"e_1_3_9_22_2","doi-asserted-by":"publisher","DOI":"10.1002\/ijc.32358"},{"key":"e_1_3_9_23_2","article-title":"Chain-of-Thought prompting elicits reasoning in large language models","author":"Wei J","year":"2022","unstructured":"Wei J, Wang X, Schuurmans D, et al: Chain-of-Thought prompting elicits reasoning in large language models. arXiv [cs.CL], 2022. http:\/\/arxiv.org\/abs\/2201.11903","journal-title":"arXiv [cs.CL]"},{"key":"e_1_3_9_24_2","unstructured":"GitHub: https:\/\/github.com\/LammertJ\/MEREDITH\/blob\/main\/README.md"},{"key":"e_1_3_9_25_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-024-45879-8"},{"key":"e_1_3_9_26_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12916-022-02560-5"},{"key":"e_1_3_9_27_2","doi-asserted-by":"publisher","DOI":"10.1007\/s12094-024-03468-7"},{"key":"e_1_3_9_28_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-023-00927-3"},{"key":"e_1_3_9_29_2","doi-asserted-by":"publisher","DOI":"10.1038\/s43856-023-00279-9"},{"key":"e_1_3_9_30_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01000-3"},{"key":"e_1_3_9_31_2","doi-asserted-by":"publisher","DOI":"10.1007\/s00146-022-01607-8"},{"key":"e_1_3_9_32_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-023-00929-1"},{"key":"e_1_3_9_33_2","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMoa2002787"},{"key":"e_1_3_9_34_2","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMoa2103695"},{"key":"e_1_3_9_35_2","article-title":"Response of NF1-mutated melanoma to an MEK inhibitor","author":"Py C","unstructured":"Py C, Christinat Y, Kreutzfeldt M, et al: Response of NF1-mutated melanoma to an MEK inhibitor. JCO Precis Oncol 10.1200\/PO.18.00028","journal-title":"JCO Precis Oncol"},{"key":"e_1_3_9_36_2","doi-asserted-by":"publisher","DOI":"10.18632\/oncotarget.22768"},{"key":"e_1_3_9_37_2","doi-asserted-by":"publisher","DOI":"10.4062\/biomolther.2021.047"},{"key":"e_1_3_9_38_2","unstructured":"ASCO: Association of STK11\/LKB1 genomic alterations with lack of benefit from the addition of pembrolizumab to platinum doublet chemotherapy in non-squamous non-small cell lung cancer. https:\/\/meetings.asco.org\/abstracts-presentations\/175038"},{"key":"e_1_3_9_39_2","unstructured":"Flaticon. https:\/\/www.flaticon.com"}],"container-title":["JCO Precision Oncology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/ascopubs.org\/doi\/pdfdirect\/10.1200\/PO-24-00478","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,10,30]],"date-time":"2024-10-30T19:59:28Z","timestamp":1730318368000},"score":1,"resource":{"primary":{"URL":"https:\/\/ascopubs.org\/doi\/10.1200\/PO-24-00478"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10]]},"references-count":38,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2024,10]]}},"alternative-id":["10.1200\/PO-24-00478"],"URL":"https:\/\/doi.org\/10.1200\/po-24-00478","relation":{},"ISSN":["2473-4284"],"issn-type":[{"value":"2473-4284","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10]]},"assertion":[{"value":"2024-07-09","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-08-30","order":1,"name":"revised","label":"Revised","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-09-27","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-10-30","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e2400478"}}