{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,1]],"date-time":"2026-01-01T14:12:05Z","timestamp":1767276725380,"version":"3.41.0"},"reference-count":35,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2025,5,23]],"date-time":"2025-05-23T00:00:00Z","timestamp":1747958400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2025,5,23]],"date-time":"2025-05-23T00:00:00Z","timestamp":1747958400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Basic Research Project of Yunnan Science and Technology Department","award":["202201AY070001-058"],"award-info":[{"award-number":["202201AY070001-058"]}]},{"name":"the\u201cXingdian Talents\u201dProject of Yunnan Province","award":["(RLMY20200020)","(RLMY20220006)"],"award-info":[{"award-number":["(RLMY20200020)","(RLMY20220006)"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Discov Onc"],"abstract":"<jats:title>Abs<\/jats:title>\n          <jats:sec>\n            <jats:title>Objective<\/jats:title>\n            <jats:p>This study aimed to compare the performance of three large language models (LLMs)\u2014ChatGPT-3.5, ChatGPT-4, and Open AI-o1\u2014in addressing clinical questions related to Programmed Cell Death in multiple myeloma. By evaluating each model's accuracy, comprehensiveness, and self-correcting capabilities, the investigation sought to determine the most effective tool for supporting clinical decision-making in this specialized oncological context.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Methods<\/jats:title>\n            <jats:p>A comprehensive set of forty clinical questions was curated from recent high-impact oncology journals, International Myeloma Working Group (IMWG) guidelines, and reputable medical databases, covering various aspects of Programmed Cell Death in multiple myeloma. These questions were refined and validated by a panel of four hematologists-oncologists with expertise in the field. Each question was individually posed to ChatGPT-3.5, ChatGPT-4, and Open AI-o1 in controlled sessions. Responses were anonymized and evaluated by the same panel using a five-point Likert scale assessing accuracy, depth, and completeness. Responses were categorized as \u201cexcellent,\u201d \u201csatisfactory,\u201d or \u201cinsufficient\u201d based on cumulative scores. Additionally, the models\u2019 self-correcting abilities were assessed by providing feedback on initially insufficient responses and re-evaluating the revised answers. Interrater reliability was measured using Cohen\u2019s Kappa coefficients.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Results<\/jats:title>\n            <jats:p>Open AI-o1 consistently generated the most extensive and detailed responses, achieving significantly higher total scores across all domains compared to ChatGPT-3.5 and ChatGPT-4. It demonstrated the lowest proportion of \"insufficient\" responses and the highest percentage of \u201cexcellent\u201d answers, particularly excelling in guideline-based questions. Open AI-o1 also exhibited superior self-correcting capacity, effectively enhancing its responses upon receiving feedback. The highest Cohen\u2019s Kappa coefficient among the models indicated greater consistency in evaluations by clinical experts. User satisfaction surveys revealed that 85% of hematologists-oncologists rated Open AI-o1 as \"highly satisfactory,\" compared to 60% for ChatGPT-4 and 45% for ChatGPT-3.5.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>Conclusion<\/jats:title>\n            <jats:p>Open AI-o1 outperforms ChatGPT-3.5 and ChatGPT-4 in accuracy, depth, and reliability when addressing complex clinical questions related to Programmed Cell Death in multiple myeloma. Its advanced \u201cthinking\u201d ability facilitates comprehensive and evidence-based responses, making it a more dependable tool for clinical decision support. These findings suggest that Open AI-o1 holds significant potential for enhancing clinical practices in specialized oncological fields, though ongoing validation and integration with human expertise remain essential.<\/jats:p>\n          <\/jats:sec>","DOI":"10.1007\/s12672-025-02648-3","type":"journal-article","created":{"date-parts":[[2025,5,23]],"date-time":"2025-05-23T11:31:11Z","timestamp":1747999871000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":1,"title":["Comparative analysis of the performance of the large language models ChatGPT-3.5, ChatGPT-4 and Open AI-o1 in the field of Programmed Cell Death in myeloma"],"prefix":"10.1007","volume":"16","author":[{"given":"Wu","family":"Kun","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Tao","family":"Bo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Li","family":"Yuntao","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Cheng","family":"Shenju","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Li","family":"Yanhong","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Luo","family":"Shan","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zeng","family":"Yun","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Nie","family":"Bo","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Shi","family":"Mingxia","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2025,5,23]]},"reference":[{"issue":"5","key":"2648_CR1","doi-asserted-by":"publisher","first-page":"464","DOI":"10.1001\/jama.2022.0003","volume":"327","author":"AJ Cowan","year":"2022","unstructured":"Cowan AJ, Green DJ, Kwok M, et al. Diagnosis and management of multiple myeloma: a review. JAMA. 2022;327(5):464\u201377.","journal-title":"JAMA"},{"issue":"2","key":"2648_CR2","doi-asserted-by":"publisher","DOI":"10.1016\/j.sjbs.2023.103920","volume":"31","author":"MS Abduh","year":"2024","unstructured":"Abduh MS. An overview of multiple myeloma: a monoclonal plasma cell malignancy\u2019s diagnosis, management, and treatment modalities. Saudi Journal of Biological Sciences. 2024;31(2): 103920.","journal-title":"Saudi Journal of Biological Sciences"},{"issue":"1","key":"2648_CR3","doi-asserted-by":"publisher","first-page":"510","DOI":"10.1080\/16078454.2021.1948208","volume":"26","author":"W Han","year":"2021","unstructured":"Han W, Jin Y, Xu M, et al. Prognostic value of circulating clonal plasma cells in newly diagnosed multiple myeloma. Hematology. 2021;26(1):510\u20137.","journal-title":"Hematology"},{"issue":"8","key":"2648_CR4","doi-asserted-by":"publisher","first-page":"1086","DOI":"10.1002\/ajh.26590","volume":"97","author":"SV Rajkumar","year":"2022","unstructured":"Rajkumar SV. Multiple myeloma: 2022 update on diagnosis, risk stratification, and management. Am J Hematol. 2022;97(8):1086\u2013107.","journal-title":"Am J Hematol"},{"key":"2648_CR5","doi-asserted-by":"publisher","DOI":"10.1016\/j.saa.2021.119813","volume":"258","author":"M Bonsignore","year":"2021","unstructured":"Bonsignore M, Trusso S, De Pasquale C, et al. A multivariate analysis of Multiple Myeloma subtype plasma cells. Spectrochim Acta Part A Mol Biomol Spectrosc. 2021;258: 119813.","journal-title":"Spectrochim Acta Part A Mol Biomol Spectrosc"},{"issue":"4","key":"2648_CR6","doi-asserted-by":"publisher","first-page":"223","DOI":"10.1038\/s41571-021-00593-y","volume":"19","author":"AK Dutta","year":"2022","unstructured":"Dutta AK, Alberge JB, Sklavenitis-Pistofidis R, et al. Single-cell profiling of tumour evolution in multiple myeloma\u2014opportunities for precision medicine. Nat Rev Clin Oncol. 2022;19(4):223\u201336.","journal-title":"Nat Rev Clin Oncol"},{"issue":"17","key":"2648_CR7","first-page":"2360","volume":"137","author":"YJ Shen","year":"2021","unstructured":"Shen YJ, Mishima Y, Shi J, et al. Progression signature underlies clonal evolution and dissemination of multiple myeloma. Blood J Am Soc Hemat. 2021;137(17):2360\u201372.","journal-title":"Blood J Am Soc Hemat"},{"issue":"2","key":"2648_CR8","doi-asserted-by":"publisher","first-page":"727","DOI":"10.1007\/s11357-022-00698-x","volume":"45","author":"VS Urban","year":"2023","unstructured":"Urban VS, Cegledi A, Mikala G. Multiple myeloma, a quintessential malignant disease of aging: a geroscience perspective on pathogenesis and treatment. Geroscience. 2023;45(2):727\u201346.","journal-title":"Geroscience"},{"key":"2648_CR9","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s11357-024-01374-y","volume":"47","author":"A Kallai","year":"2024","unstructured":"Kallai A, Ungvari Z, Fekete M, et al. Genomic instability and genetic heterogeneity in aging: insights from clonal hematopoiesis (CHIP), monoclonal gammopathy (MGUS), and monoclonal B-cell lymphocytosis (MBL). Geroscience. 2024;47:1\u201318.","journal-title":"Geroscience"},{"key":"2648_CR10","first-page":"1","volume":"24","author":"SDM dos Cordas","year":"2024","unstructured":"dos Cordas SDM, Toenges R, Bertamini L, et al. New horizons in our understanding of precursor multiple myeloma and early interception. Nat Rev Cancer. 2024;24:1\u201320.","journal-title":"Nat Rev Cancer"},{"key":"2648_CR11","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1007\/s10585-024-10305-2","volume":"41","author":"MH Elbahoty","year":"2024","unstructured":"Elbahoty MH, Papineni B, Samant RS. Multiple myeloma: clinical characteristics, current therapies and emerging innovative treatments targeting ribosome biogenesis dynamics. Clin Exp Metastasis. 2024;41:1\u201314.","journal-title":"Clin Exp Metastasis"},{"key":"2648_CR12","doi-asserted-by":"publisher","first-page":"2328","DOI":"10.1182\/blood-2023-172710","volume":"142","author":"E Malek","year":"2023","unstructured":"Malek E, Wang GM, Madabhushi A, et al. Toward AI-assisted clinical assessment for patients with multiple myeloma: feature selection for large language models. Blood. 2023;142:2328.","journal-title":"Blood"},{"issue":"1","key":"2648_CR13","doi-asserted-by":"publisher","first-page":"5825","DOI":"10.1038\/s41467-023-41562-6","volume":"14","author":"C Botta","year":"2023","unstructured":"Botta C, Perez C, Larrayoz M, et al. Large T cell clones expressing immune checkpoints increase during multiple myeloma evolution and predict treatment resistance. Nat Commun. 2023;14(1):5825.","journal-title":"Nat Commun"},{"issue":"40","key":"2648_CR14","doi-asserted-by":"publisher","first-page":"2405861","DOI":"10.1002\/advs.202405861","volume":"11","author":"X Liu","year":"2024","unstructured":"Liu X, Wang Q, Zhou M, et al. DrugFormer: graph-enhanced language model to predict drug sensitivity. Adv Sci. 2024;11(40):2405861.","journal-title":"Adv Sci"},{"issue":"16_suppl","key":"2648_CR15","doi-asserted-by":"publisher","first-page":"e23196","DOI":"10.1200\/JCO.2024.42.16_suppl.e23196","volume":"42","author":"K Thorlund","year":"2024","unstructured":"Thorlund K, Lloyd-Price L, Jafar R, et al. Screening oncology articles in a qualitative literature review using large language models: a comparison of GPT4 versus fine-tuned open source models using expert-annotated data. J Clin Oncol. 2024;42(16_suppl):e23196\u2013e23196.","journal-title":"J Clin Oncol"},{"issue":"2","key":"2648_CR16","doi-asserted-by":"publisher","first-page":"375","DOI":"10.1093\/jamia\/ocad218","volume":"31","author":"S Datta","year":"2024","unstructured":"Datta S, Lee K, Paek H, et al. AutoCriteria: a generalizable clinical trial eligibility criteria extraction system powered by large language models. J Am Med Inform Assoc. 2024;31(2):375\u201385.","journal-title":"J Am Med Inform Assoc"},{"issue":"1","key":"2648_CR17","doi-asserted-by":"publisher","DOI":"10.1016\/j.acpath.2023.100101","volume":"11","author":"S Arvisais-Anhalt","year":"2024","unstructured":"Arvisais-Anhalt S, Gonias SL, Murray SG. Establishing priorities for implementation of large language models in pathology and laboratory medicine. Acad Pathol. 2024;11(1): 100101.","journal-title":"Acad Pathol"},{"issue":"1","key":"2648_CR18","doi-asserted-by":"publisher","first-page":"728","DOI":"10.1186\/s12967-023-04576-8","volume":"21","author":"M Toufiq","year":"2023","unstructured":"Toufiq M, Rinchai D, Bettacchioli E, et al. Harnessing large language models (LLMs) for candidate gene prioritization and selection. J Transl Med. 2023;21(1):728.","journal-title":"J Transl Med"},{"issue":"1","key":"2648_CR19","doi-asserted-by":"publisher","first-page":"134","DOI":"10.1038\/s42004-024-01220-4","volume":"7","author":"SH Snyder","year":"2024","unstructured":"Snyder SH, Vignaux PA, Ozalp MK, et al. The Goldilocks paradigm: comparing classical machine learning, large language models, and few-shot learning for drug discovery applications. Commun Chem. 2024;7(1):134.","journal-title":"Commun Chem"},{"issue":"3","key":"2648_CR20","doi-asserted-by":"publisher","first-page":"e213497","DOI":"10.1001\/jamanetworkopen.2021.3497","volume":"4","author":"HM Blommestein","year":"2021","unstructured":"Blommestein HM, Franken MG, van Beurden-Tan CHY, et al. Cost-effectiveness of novel treatment sequences for transplant-ineligible patients with multiple myeloma. JAMA Netw Open. 2021;4(3):e213497\u2013e213497.","journal-title":"JAMA Netw Open"},{"issue":"1","key":"2648_CR21","first-page":"162","volume":"31","author":"GL Liu","year":"2023","unstructured":"Liu GL, Chen XM, Zhang JD, et al. Construction of a prognostic model of multiple myeloma based on metabolism-related genes. Zhongguo Shi Yan Xue Ye Xue Za Zhi. 2023;31(1):162\u20139.","journal-title":"Zhongguo Shi Yan Xue Ye Xue Za Zhi"},{"key":"2648_CR22","doi-asserted-by":"publisher","DOI":"10.1021\/acs.jcim.4c01966","author":"A Gangwal","year":"2024","unstructured":"Gangwal A, Lavecchia A. AI-driven drug discovery for rare diseases. J Chem Inf Model. 2024. https:\/\/doi.org\/10.1021\/acs.jcim.4c01966.","journal-title":"J Chem Inf Model"},{"issue":"1","key":"2648_CR23","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1038\/s41598-024-71358-7","volume":"14","author":"VK Prasad","year":"2024","unstructured":"Prasad VK, Verma A, Bhattacharya P, et al. Revolutionizing healthcare: a comparative insight into deep learning\u2019s role in medical imaging. Sci Rep. 2024;14(1):1\u201339.","journal-title":"Sci Rep"},{"key":"2648_CR24","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2023.104338","volume":"141","author":"F Cremonesi","year":"2023","unstructured":"Cremonesi F, Planat V, Kalokyri V, et al. The need for multimodal health data modeling: a practical approach for a federated-learning healthcare platform. J Biomed Inform. 2023;141: 104338.","journal-title":"J Biomed Inform"},{"key":"2648_CR25","unstructured":"Faupel-Badger J, Kohaar I, Bahl M, et al. Defining precancer: a grand challenge for the cancer community. Nat Rev Cancer, 2024: 1\u201318."},{"key":"2648_CR26","doi-asserted-by":"publisher","first-page":"129","DOI":"10.1093\/jamia\/ocae129","volume":"31","author":"Y Li","year":"2024","unstructured":"Li Y, Zhao J, Li M, et al. RefAI: a GPT-powered retrieval-augmented generative tool for biomedical literature recommendation and summarization. J Am Med Inf Assoc. 2024;31:129.","journal-title":"J Am Med Inf Assoc"},{"issue":"1","key":"2648_CR27","doi-asserted-by":"publisher","first-page":"bbae640","DOI":"10.1093\/bib\/bbae640","volume":"26","author":"A Ultsch","year":"2025","unstructured":"Ultsch A, L\u00f6tsch J. Augmenting small biomedical datasets using generative AI methods based on self-organizing neural networks. Brief Bioinf. 2025;26(1):bbae640.","journal-title":"Brief Bioinf"},{"key":"2648_CR28","doi-asserted-by":"publisher","DOI":"10.1016\/j.neo.2024.100974","volume":"50","author":"X Zhou","year":"2024","unstructured":"Zhou X, He R, Hu WX, et al. Targeting myeloma metabolism: How abnormal metabolism contributes to multiple myeloma progression and resistance to proteasome inhibitors. Neoplasia. 2024;50: 100974.","journal-title":"Neoplasia"},{"issue":"7","key":"2648_CR29","doi-asserted-by":"publisher","first-page":"6019","DOI":"10.3390\/ijms24076019","volume":"24","author":"H Bashiri","year":"2023","unstructured":"Bashiri H, Tabatabaeian H. Autophagy: a potential therapeutic target to tackle drug resistance in multiple myeloma. Int J Mol Sci. 2023;24(7):6019.","journal-title":"Int J Mol Sci"},{"issue":"9","key":"2648_CR30","doi-asserted-by":"publisher","first-page":"929","DOI":"10.1002\/cac2.12591","volume":"44","author":"F Zhang","year":"2024","unstructured":"Zhang F, Guo J, Yu S, et al. Cellular senescence and metabolic reprogramming: unraveling the intricate crosstalk in the immunosuppressive tumor microenvironment. Cancer Commun. 2024;44(9):929\u201366.","journal-title":"Cancer Commun"},{"key":"2648_CR31","unstructured":"Hu H, Shang Y, Xu G, et al. Can GPT-O1 Kill All Bugs? An Evaluation of GPT-Family LLMs on QuixBugs. arXiv preprint arXiv:2409.10033, 2024."},{"key":"2648_CR32","unstructured":"Huang Z, Zou H, Li X, et al. O1 Replication Journey--Part 2: Surpassing O1-preview through Simple Distillation, Big Progress or Bitter Lesson?. arXiv preprint arXiv:2411.16489, 2024."},{"issue":"5","key":"2648_CR33","doi-asserted-by":"publisher","first-page":"297","DOI":"10.1007\/s00520-023-07759-7","volume":"31","author":"L Zeng","year":"2023","unstructured":"Zeng L, Huang H, Liu Y, et al. The core symptom in multiple myeloma patients undergoing chemotherapy: a network analysis. Support Care Cancer. 2023;31(5):297.","journal-title":"Support Care Cancer"},{"issue":"2","key":"2648_CR34","doi-asserted-by":"publisher","first-page":"e143","DOI":"10.1016\/S2352-3026(21)00283-0","volume":"9","author":"NS Raje","year":"2022","unstructured":"Raje NS, Anaissie E, Kumar SK, et al. Consensus guidelines and recommendations for infection prevention in multiple myeloma: a report from the international myeloma working group. Lancet Haematol. 2022;9(2):e143\u201361.","journal-title":"Lancet Haematol"},{"issue":"1","key":"2648_CR35","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1038\/s41408-023-00877-9","volume":"13","author":"MV Mateos","year":"2023","unstructured":"Mateos MV, Ailawadhi S, Costa LJ, et al. Global disparities in patients with multiple myeloma: a rapid evidence assessment. Blood Cancer J. 2023;13(1):109.","journal-title":"Blood Cancer J"}],"container-title":["Discover Oncology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12672-025-02648-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s12672-025-02648-3\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s12672-025-02648-3.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,5,23]],"date-time":"2025-05-23T11:31:15Z","timestamp":1747999875000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s12672-025-02648-3"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2025,5,23]]},"references-count":35,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2025,12]]}},"alternative-id":["2648"],"URL":"https:\/\/doi.org\/10.1007\/s12672-025-02648-3","relation":{},"ISSN":["2730-6011"],"issn-type":[{"value":"2730-6011","type":"electronic"}],"subject":[],"published":{"date-parts":[[2025,5,23]]},"assertion":[{"value":"17 January 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"9 May 2025","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 May 2025","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"This study involved expert evaluators who assessed the outputs of large language models. It did not collect or analyze personal data from patients or other human participants. The Yunnan Province Clinical Research Center for Laboratory Medicine Ethics Committee reviewed the study protocol and determined that it did not constitute human-subject research requiring formal approval. All expert evaluators provided written informed consent to participate in this assessment and consented to the publication of anonymized data, in accordance with relevant regulations and institutional guidelines.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Informed consent was obtained from all patients included in this study prior to submission of data.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"The authors declare no competing interests.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"870"}}