{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,15]],"date-time":"2026-05-15T06:29:17Z","timestamp":1778826557644,"version":"3.51.4"},"reference-count":54,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,7,2]],"date-time":"2025-07-02T00:00:00Z","timestamp":1751414400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,7,2]],"date-time":"2025-07-02T00:00:00Z","timestamp":1751414400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100000054","name":"U.S. Department of Health & Human Services | NIH | National Cancer Institute","doi-asserted-by":"publisher","award":["P30 CA008748"],"award-info":[{"award-number":["P30 CA008748"]}],"id":[{"id":"10.13039\/100000054","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["npj Digit. Med."],"abstract":"<jats:title>Abstract<\/jats:title>\n          <jats:p>Subspecialty knowledge barriers have limited the adoption of large language models (LLMs) in oncology. We introduce Woollie, an open-source, oncology-specific LLM trained on real-world data from Memorial Sloan Kettering Cancer Center (MSK) across lung, breast, prostate, pancreatic, and colorectal cancers, with external validation using University of California, San Francisco (UCSF) data. Woollie surpasses ChatGPT in medical benchmarks and excels in eight non-medical benchmarks. Analyzing 39,319 radiology impression notes from 4002 patients, it achieved an overall area under the receiver operating characteristic curve (AUROC) of 0.97 for cancer progression prediction on MSK data, including a notable 0.98 AUROC for pancreatic cancer. On UCSF data, it achieved an overall AUROC of 0.88, excelling in lung cancer detection with an AUROC of 0.95. As the first oncology specific LLM validated across institutions, Woollie demonstrates high accuracy and consistency across cancer types, underscoring its potential to enhance cancer progression analysis.<\/jats:p>","DOI":"10.1038\/s41746-025-01780-2","type":"journal-article","created":{"date-parts":[[2025,7,2]],"date-time":"2025-07-02T06:44:45Z","timestamp":1751438685000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":18,"title":["Large language model trained on clinical oncology data predicts cancer progression"],"prefix":"10.1038","volume":"8","author":[{"given":"Menglei","family":"Zhu","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7356-3611","authenticated-orcid":false,"given":"Hui","family":"Lin","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jue","family":"Jiang","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Abbas J.","family":"Jinia","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Justin","family":"Jee","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Karl","family":"Pichotta","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5416-5172","authenticated-orcid":false,"given":"Michele","family":"Waters","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Doori","family":"Rose","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nikolaus","family":"Schultz","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sulov","family":"Chalise","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lohit","family":"Valleru","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9240-5704","authenticated-orcid":false,"given":"Olivier","family":"Morin","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jean","family":"Moran","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Joseph O.","family":"Deasy","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shirin","family":"Pilai","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Chelsea","family":"Nichols","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gregory","family":"Riely","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lior Z.","family":"Braunstein","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1238-2072","authenticated-orcid":false,"given":"Anyi","family":"Li","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2025,7,2]]},"reference":[{"key":"1780_CR1","first-page":"27730","volume":"35","author":"L Ouyang","year":"2022","unstructured":"Ouyang, L. et al. Training language models to follow instructions with human feedback. Adv. Neural Inf. Process. Syst. 35, 27730\u201327744 (2022).","journal-title":"Adv. Neural Inf. Process. Syst."},{"key":"1780_CR2","unstructured":"OpenAi. GPT-4 technical report (OpenAi, 2023)."},{"key":"1780_CR3","unstructured":"Eloundou, T., Manning, S., Mishkin, P. & Rock, D. Gpts are gpts: an early look at the labor market impact potential of large language models. Preprint at https:\/\/arxiv.org\/abs\/2303.10130 (2023)."},{"key":"1780_CR4","doi-asserted-by":"crossref","unstructured":"Will ChatGPT transform healthcare? Nat. Med. 29, 505\u2013506 (2023)","DOI":"10.1038\/s41591-023-02289-5"},{"key":"1780_CR5","unstructured":"Nori, H., King, N., McKinney, S. M., Carignan, D. & Horvitz, E. Capabilities of gpt-4 on medical challenge problems. Preprint at https:\/\/arxiv.org\/abs\/2303.13375 (2023)."},{"key":"1780_CR6","doi-asserted-by":"publisher","first-page":"e0000198","DOI":"10.1371\/journal.pdig.0000198","volume":"2","author":"TH Kung","year":"2023","unstructured":"Kung, T. H. et al. Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models. PLOS Digital Health 2, e0000198 (2023).","journal-title":"PLOS Digital Health"},{"key":"1780_CR7","first-page":"e35237","volume":"15","author":"RK Sinha","year":"2023","unstructured":"Sinha, R. K., Deb Roy, A., Kumar, N. & Mondal, H. Applicability of ChatGPT in assisting to solve higher order problems in pathology. Cureus 15, e35237 (2023).","journal-title":"Cureus"},{"key":"1780_CR8","doi-asserted-by":"publisher","unstructured":"Rao, A. et al. Evaluating ChatGPT as an adjunct for radiologic decision-making. medRxiv https:\/\/doi.org\/10.1101\/2023.02.02.23285399 (2023)","DOI":"10.1101\/2023.02.02.23285399"},{"key":"1780_CR9","doi-asserted-by":"publisher","unstructured":"Chen, S. et al. Use of artificial intelligence chatbots for cancer treatment information. JAMA Oncol. https:\/\/doi.org\/10.1001\/jamaoncol.2023.2954 (2023)","DOI":"10.1001\/jamaoncol.2023.2954"},{"key":"1780_CR10","unstructured":"Chung, H. W. et al. Scaling instruction-finetuned language models. J. Mach. Learn. Res 25.70 1\u201353 (2024)."},{"key":"1780_CR11","unstructured":"Touvron, H. et al. Llama: open and efficient foundation language models. Preprint at https:\/\/arxiv.org\/abs\/2302.13971 (2023)."},{"key":"1780_CR12","unstructured":"Touvron, H. et al. Llama 2: Open foundation and fine-tuned chat models. Preprint at https:\/\/arxiv.org\/abs\/2307.09288 (2023)."},{"key":"1780_CR13","unstructured":"Bolton, E. et al. Biomedlm: A 2.7 b parameter language model trained on biomedical text. Preprint at https:\/\/arxiv.org\/abs\/2403.18421 (2024)."},{"key":"1780_CR14","doi-asserted-by":"publisher","unstructured":"Luo, R. et al. BioGPT: generative pre-trained transformer for biomedical text generation and mining. Brief. Bioinformatics 23, https:\/\/doi.org\/10.1093\/bib\/bbac409 (2022).","DOI":"10.1093\/bib\/bbac409"},{"key":"1780_CR15","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-022-00742-2","volume":"5","author":"X Yang","year":"2022","unstructured":"Yang, X. et al. A large language model for electronic health records. npj Digital Med. 5, 194. https:\/\/doi.org\/10.1038\/s41746-022-00742-2 (2022).","journal-title":"npj Digital Med."},{"key":"1780_CR16","unstructured":"Singhal, K. et al. Large language models encode clinical knowledge. Preprint at https:\/\/arxiv.org\/abs\/2212.13138 (2022)."},{"key":"1780_CR17","unstructured":"Singhal, K. et al. Toward expert-level medical question answering with large language models. Nat. Med 1\u20138 (2025)."},{"key":"1780_CR18","doi-asserted-by":"crossref","unstructured":"Labrak, Y. et al. Biomistral: a collection of open-source pretrained large language models for medical domains. Preprint at https:\/\/arxiv.org\/abs\/2402.10373 (2024).","DOI":"10.18653\/v1\/2024.findings-acl.348"},{"key":"1780_CR19","unstructured":"Jin, Q., Dhingra, B., Liu, Z., Cohen, W. & Lu, X. 2567\u20132577 (2019)."},{"key":"1780_CR20","unstructured":"Pal, A., Logesh, K. U. & Malaikannan, S. Medmcqa: A large-scale multi-subject multi-choice dataset for medical domain question answering. Conference on health, inference, and learning. (PMLR, 2022)."},{"key":"1780_CR21","unstructured":"Mihaylov, T., Clark, P., Khot, T. & Sabharwal, A. in Conference on Empirical Methods in Natural Language Processing."},{"key":"1780_CR22","doi-asserted-by":"crossref","unstructured":"Liu, J. et al. LogiQA: a challenge dataset for machine reading comprehension with logical reasoning. Preprint at https:\/\/arxiv.org\/abs\/2007.08124 (2020).","DOI":"10.24963\/ijcai.2020\/501"},{"key":"1780_CR23","unstructured":"Hendrycks, D. et al. Measuring massive multitask language understanding. Preprint at https:\/\/arxiv.org\/abs\/2009.03300 (2020)."},{"key":"1780_CR24","doi-asserted-by":"publisher","first-page":"249","DOI":"10.1162\/tacl_a_00266","volume":"7","author":"S Reddy","year":"2019","unstructured":"Reddy, S., Chen, D. & Manning, C. D. Coqa: a conversational question answering challenge. Trans. Assoc. Computational Linguist. 7, 249\u2013266 (2019).","journal-title":"Trans. Assoc. Computational Linguist."},{"key":"1780_CR25","doi-asserted-by":"crossref","unstructured":"Zellers, R., Holtzman, A., Bisk, Y., Farhadi, A. & Choi, Y. Hellaswag: Can a machine really finish your sentence? Preprint at https:\/\/arxiv.org\/abs\/1905.07830 (2019).","DOI":"10.18653\/v1\/P19-1472"},{"key":"1780_CR26","unstructured":"Luo, Y. et al. An empirical study of catastrophic forgetting in large language models during continual fine-tuning. Preprint at https:\/\/arxiv.org\/html\/2308.08747v3 (2023)."},{"key":"1780_CR27","unstructured":"Brown, T. et al. (eds H. Larochelle et al.) 1877-1901 (Curran Associates, Inc., 2020)."},{"key":"1780_CR28","unstructured":"Achiam, J. et al. Gpt-4 technical report. Preprint at https:\/\/arxiv.org\/abs\/2303.08774 (2023)."},{"key":"1780_CR29","unstructured":"Bureau, U. S. C. United States Census Bureau Quick Facts. https:\/\/www.census.gov\/quickfacts\/fact\/table\/US\/PST045224 (2024)."},{"key":"1780_CR30","doi-asserted-by":"publisher","first-page":"5721","DOI":"10.1158\/1538-7445.AM2023-5721","volume":"83","author":"J Jee","year":"2023","unstructured":"Jee, J. et al. Automated annotation for large-scale clinicogenomic models of lung cancer treatment response and overall survival. Cancer Res. 83, 5721\u20135721 (2023).","journal-title":"Cancer Res."},{"key":"1780_CR31","first-page":"993","volume":"3","author":"DM Blei","year":"2003","unstructured":"Blei, D. M., Ng, A. Y. & Jordan, M. I. Latent dirichlet allocation. J. Mach. Learn. Res. 3, 993\u20131022 (2003).","journal-title":"J. Mach. Learn. Res."},{"key":"1780_CR32","doi-asserted-by":"publisher","first-page":"3271","DOI":"10.1200\/jco.2009.25.9820","volume":"28","author":"H Kennecke","year":"2010","unstructured":"Kennecke, H. et al. Metastatic behavior of breast cancer subtypes. J. Clin. Oncol. 28, 3271\u20133277, https:\/\/doi.org\/10.1200\/jco.2009.25.9820 (2010).","journal-title":"J. Clin. Oncol."},{"key":"1780_CR33","doi-asserted-by":"publisher","first-page":"38","DOI":"10.1097\/00007611-199301000-00009","volume":"86","author":"V Patanaphan","year":"1993","unstructured":"Patanaphan, V. & Salazar, O. M. Colorectal cancer: metastatic patterns and prognosis. South Med. J. 86, 38\u201341 (1993).","journal-title":"South Med. J."},{"key":"1780_CR34","unstructured":"Ribeiro, M. T., Singh, S. & Guestrin, C. in Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 1135\u20131144."},{"key":"1780_CR35","doi-asserted-by":"publisher","first-page":"115","DOI":"10.1148\/radiol.2021210043","volume":"301","author":"RKG Do","year":"2021","unstructured":"Do, R. K. G. et al. Patterns of metastatic disease in patients with cancer derived from natural language processing of structured CT radiology reports over a 10-year period. Radiology 301, 115\u2013122(2021).","journal-title":"Radiology"},{"key":"1780_CR36","unstructured":"Dettmers, T., Pagnoni, A., Holtzman, A. & Zettlemoyer, L. Qlora: Efficient finetuning of quantized LLMs. In Advances in neural information processing Systems 36 (NIPS, 2024)."},{"key":"1780_CR37","unstructured":"Chiang, W.-L. et al. Vicuna: an open-source chatbot impressing GPT-4 with 90%* ChatGPT quality (2023)."},{"key":"1780_CR38","unstructured":"Wang, Y. et al. Self-instruct: aligning language model with self generated instructions. Preprint at https:\/\/arxiv.org\/abs\/2212.10560 (2022)."},{"key":"1780_CR39","unstructured":"Gene, R. Alpaca dataset from Stanford, cleaned and curated (https:\/\/github.com\/gururise\/AlpacaDataCleaned) 2023."},{"key":"1780_CR40","unstructured":"Si, Q. et al. An empirical study of instruction-tuning large language models in Chinese. Preprint at https:\/\/arxiv.org\/abs\/2310.07328 (2023)."},{"key":"1780_CR41","unstructured":"Xue, F., Jain, K., Shah, M. H., Zheng, Z. & Yang, Y. Instruction in the wild: a user-based instruction dataset (https:\/\/github.com\/XueFuzhao\/InstructionWild) 2023."},{"key":"1780_CR42","unstructured":"Dettmers, T. et al. Qlora: Efficient finetuning of quantized llms. Advances in neural information processing systems 36, 10088\u201310115 (2023)."},{"key":"1780_CR43","doi-asserted-by":"crossref","unstructured":"Jin, Q., Dhingra, B., Liu, Z., Cohen, W. W. & Lu, X. Pubmedqa: a dataset for biomedical research question answering. Preprint at https:\/\/arxiv.org\/abs\/1909.06146 (2019).","DOI":"10.18653\/v1\/D19-1259"},{"key":"1780_CR44","doi-asserted-by":"publisher","DOI":"10.1186\/s12859-019-3119-4","volume":"20","author":"A Ben Abacha","year":"2019","unstructured":"Ben Abacha, A. & Demner-Fushman, D. A question-entailment approach to question answering. BMC Bioinforma. 20, 511(2019).","journal-title":"BMC Bioinforma."},{"key":"1780_CR45","doi-asserted-by":"publisher","first-page":"e2100105","DOI":"10.1200\/CCI.21.00105","volume":"6","author":"JA Lavery","year":"2022","unstructured":"Lavery, J. A. et al. A scalable quality assurance process for curating oncology electronic health records: the project GENIE biopharma collaborative approach. JCO Clin. Cancer Inf. 6, e2100105(2022).","journal-title":"JCO Clin. Cancer Inf."},{"key":"1780_CR46","unstructured":"Schrag, D. in ASCO annual meeting."},{"key":"1780_CR47","doi-asserted-by":"publisher","first-page":"157","DOI":"10.1162\/tacl_a_00638","volume":"12","author":"NF Liu","year":"2024","unstructured":"Liu, N. F. et al. Lost in the middle: how language models use long contexts. Trans. Assoc. Computational Linguist. 12, 157\u2013173 (2024).","journal-title":"Trans. Assoc. Computational Linguist."},{"key":"1780_CR48","unstructured":"Liu, Y. RoBERTa: a robustly optimized bert pretraining approach. Preprint at https:\/\/arxiv.org\/abs\/1907.11692 (2019)."},{"key":"1780_CR49","unstructured":"Honnibal, M., Montani, I., Van Landeghem, S. & Boyd, A. spaCy: industrial-strength natural language processing in python. https:\/\/arxiv.org\/abs\/1907.11692 (2020)."},{"key":"1780_CR50","unstructured":"Gao, L. et al. A framework for few-shot language model evaluation (Zenodo, 2021)."},{"key":"1780_CR51","unstructured":"Grootendorst, M. BERTopic: Neural topic modeling with a class-based TF-IDF procedure. Preprint at https:\/\/arxiv.org\/abs\/2203.05794 (2022)."},{"key":"1780_CR52","doi-asserted-by":"crossref","unstructured":"Xiao, S. et al. C-pack: Packed resources for general chinese embeddings. Proceedings of the 47th international ACM SIGIR conference on research and development in information retrieval. (2024).","DOI":"10.1145\/3626772.3657878"},{"key":"1780_CR53","doi-asserted-by":"crossref","unstructured":"McInnes, L., Healy, J. & Melville, J. Umap: uniform manifold approximation and projection for dimension reduction. Preprint at https:\/\/arxiv.org\/abs\/1802.03426 (2018).","DOI":"10.21105\/joss.00861"},{"key":"1780_CR54","doi-asserted-by":"publisher","unstructured":"Campello, R., Moulavi, D. & Sander, J. (2013). Density-Based Clustering Based on Hierarchical Density Estimates. 7819, 160\u2013172. https:\/\/doi.org\/10.1007\/978-3-642-37456-2_14.","DOI":"10.1007\/978-3-642-37456-2_14"}],"container-title":["npj Digital Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.nature.com\/articles\/s41746-025-01780-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-025-01780-2","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.nature.com\/articles\/s41746-025-01780-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,2]],"date-time":"2025-07-02T06:45:02Z","timestamp":1751438702000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.nature.com\/articles\/s41746-025-01780-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,7,2]]},"references-count":54,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["1780"],"URL":"https:\/\/doi.org\/10.1038\/s41746-025-01780-2","relation":{},"ISSN":["2398-6352"],"issn-type":[{"value":"2398-6352","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,7,2]]},"assertion":[{"value":"23 July 2024","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"4 June 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"2 July 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"The authors declare no competing interests.","order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"397"}}