{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,8,18]],"date-time":"2026-08-18T04:38:47Z","timestamp":1787027927960,"version":"build-2736575974"},"reference-count":33,"publisher":"MDPI AG","issue":"19","license":[{"start":{"date-parts":[[2023,10,5]],"date-time":"2023-10-05T00:00:00Z","timestamp":1696464000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["11901013"],"award-info":[{"award-number":["11901013"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["1204031"],"award-info":[{"award-number":["1204031"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["RDX2021-05"],"award-info":[{"award-number":["RDX2021-05"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004826","name":"Beijing Natural Science Foundation","doi-asserted-by":"publisher","award":["11901013"],"award-info":[{"award-number":["11901013"]}],"id":[{"id":"10.13039\/501100004826","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004826","name":"Beijing Natural Science Foundation","doi-asserted-by":"publisher","award":["1204031"],"award-info":[{"award-number":["1204031"]}],"id":[{"id":"10.13039\/501100004826","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100004826","name":"Beijing Natural Science Foundation","doi-asserted-by":"publisher","award":["RDX2021-05"],"award-info":[{"award-number":["RDX2021-05"]}],"id":[{"id":"10.13039\/501100004826","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Scientific Research and Development Funds of Peking University People\u2019s Hospital","award":["11901013"],"award-info":[{"award-number":["11901013"]}]},{"name":"Scientific Research and Development Funds of Peking University People\u2019s Hospital","award":["1204031"],"award-info":[{"award-number":["1204031"]}]},{"name":"Scientific Research and Development Funds of Peking University People\u2019s Hospital","award":["RDX2021-05"],"award-info":[{"award-number":["RDX2021-05"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Diagnostics"],"abstract":"<jats:p>Purpose: By incorporating the cost of multiple tumor-marker tests, this work aims to comprehensively evaluate the financial burden of patients and the accuracy of machine learning models in diagnosing malignant pleural effusion (MPE) using tumor-marker combinations. Methods: Carcinoembryonic antigen (CEA), carbohydrate antigen (CA)19-9, CA125, and CA15-3 were collected from pleural effusion (PE) and peripheral blood (PB) of 319 patients with pleural effusion. A stacked ensemble (stacking) model based on five machine learning models was utilized to evaluate the diagnostic accuracy of tumor markers. We evaluated the discriminatory accuracy of various tumor-marker combinations using the area under the curve (AUC), sensitivity, and specificity. To evaluate the cost-effectiveness of different tumor-marker combinations, a comprehensive score (C-score) with a tuning parameter w was proposed. Results: In most scenarios, the stacking model outperformed the five individual machine learning models in terms of AUC. Among the eight tumor markers, the CEA in PE (PE.CEA) showed the best AUC of 0.902. Among all tumor-marker combinations, the PE.CA19-9 + PE.CA15-3 + PE.CEA + PB.CEA combination (C9 combination) achieved the highest AUC of 0.946. When w puts more weight on the cost, the highest C-score was achieved with the single PE.CEA marker. As w puts over 0.8 weight on AUC, the C-score favored diagnostic models with more expensive tumor-marker combinations. Specifically, when w was set to 0.99, the C9 combination achieved the best C-score. Conclusion: The stacking diagnostic model using PE.CEA is a relatively accurate and affordable choice in diagnosing MPE for patients without medical insurance or in a low economic level. The stacking model using the combination PE.CA19-9 + PE.CA15-3 + PE.CEA + PB.CEA is the most accurate diagnostic model and the best choice for patients without an economic burden. From a cost-effectiveness perspective, the stacking diagnostic model with PE.CA19-9 + PE.CA15-3 + PE.CEA combination is particularly recommended, as it gains the best trade-off between the low cost and high effectiveness.<\/jats:p>","DOI":"10.3390\/diagnostics13193136","type":"journal-article","created":{"date-parts":[[2023,10,6]],"date-time":"2023-10-06T01:07:13Z","timestamp":1696554433000},"page":"3136","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":8,"title":["The Diagnosis of Malignant Pleural Effusion Using Tumor-Marker Combinations: A Cost-Effectiveness Analysis Based on a Stacking Model"],"prefix":"10.3390","volume":"13","author":[{"ORCID":"https:\/\/orcid.org\/0009-0009-8460-770X","authenticated-orcid":false,"given":"Jingyuan","family":"Wang","sequence":"first","affiliation":[{"name":"Department of Biostatistics, School of Public Health, Peking University, Beijing 100191, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jiangjie","family":"Zhou","sequence":"additional","affiliation":[{"name":"Department of Biostatistics, School of Public Health, Peking University, Beijing 100191, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hanyu","family":"Wu","sequence":"additional","affiliation":[{"name":"Department of Biostatistics, School of Public Health, Peking University, Beijing 100191, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Yangyu","family":"Chen","sequence":"additional","affiliation":[{"name":"Department of Respiration and Critical Care Medicine, Beijing Chaoyang Hospital, Beijing 100020, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3500-0820","authenticated-orcid":false,"given":"Baosheng","family":"Liang","sequence":"additional","affiliation":[{"name":"Department of Biostatistics, School of Public Health, Peking University, Beijing 100191, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2023,10,5]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1097\/PAT.0000000000000215","article-title":"Diagnostic performance of CD66c in lung adenocarcinoma-associated malignant pleural effusion: Comparison with CEA, CA 19-9, and CYFRA 21-1","volume":"47","author":"Son","year":"2015","journal-title":"Pathology"},{"key":"ref_2","first-page":"533","article-title":"The Value of Multiple Preparations in the Diagnosis of Malignant Pleural Effusions\u2014A Cost-Benefit-Analysis","volume":"35","author":"Starr","year":"1991","journal-title":"Acta Cytol."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"505","DOI":"10.7314\/APJCP.2012.13.2.505","article-title":"Application of MMP-7 and MMP-10 in Assisting the Diagnosis of Malignant Pleural Effusion","volume":"13","author":"Cheng","year":"2012","journal-title":"Asian Pac. J. Cancer Prev."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"991","DOI":"10.1378\/chest.13-2481","article-title":"Comparing Cost of Indwelling Pleural Catheter vs Talc Pleurodesis for Malignant Pleural Effusion","volume":"146","author":"Penz","year":"2014","journal-title":"Chest"},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"374","DOI":"10.1016\/j.athoracsur.2012.02.100","article-title":"Treatment of Malignant Pleural Effusion: A Cost-Effectiveness Analysis","volume":"94","author":"Puri","year":"2012","journal-title":"Ann. Thorac. Surg."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.lungcan.2006.04.016","article-title":"Malignant pleural effusion, current and evolving approaches for its diagnosis and management","volume":"54","year":"2006","journal-title":"Lung Cancer"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1016\/0305-7372(93)90032-M","article-title":"Diagnosis and management of malignant pleural effusion","volume":"19","author":"Miles","year":"1993","journal-title":"Cancer Treat. Rev."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"211","DOI":"10.4046\/trd.2014.76.5.211","article-title":"Malignant Pleural Effusion: Medical Approaches for Diagnosis and Management","volume":"76","year":"2014","journal-title":"Tuberc. Respir. Dis."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"420","DOI":"10.1097\/COC.0b013e3181aacbbf","article-title":"Diagnosis and treatment of malignant pleural effusion: A systematic literature review and new approaches","volume":"33","author":"Lombardi","year":"2010","journal-title":"Am. J. Clin. Oncol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"839","DOI":"10.1164\/rccm.201807-1415ST","article-title":"Management of Malignant Pleural Effusions. An Official ATS\/STS\/STR Clinical Practice Guideline","volume":"198","author":"Reddy","year":"2018","journal-title":"Am. J. Resp. Crit. Care"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"559","DOI":"10.1055\/s-0042-1748185","article-title":"Management of Malignant Pleural Effusion: Where Are We Now?","volume":"43","author":"Dutau","year":"2022","journal-title":"Semin. Respir. Crit. Care Med."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1159\/000468545","article-title":"Diagnostic Accuracy of Combinations of Tumor Markers for Malignant Pleural Effusion: An Updated Meta-Analysis","volume":"94","author":"Yang","year":"2017","journal-title":"Respiration"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1557","DOI":"10.21037\/tlcr-20-1111","article-title":"Pleural biomarkers in diagnostics of malignant pleural effusion: A narrative review","volume":"10","author":"Zhang","year":"2021","journal-title":"Transl. Lung Cancer Res."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"139","DOI":"10.1177\/03936155231158125","article-title":"Diagnosis of malignant pleural effusion with combinations of multiple tumor markers: A comparison study of five machine learning models","volume":"38","author":"Zhang","year":"2023","journal-title":"Int. J. Biol. Marker"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"59","DOI":"10.1089\/jpm.2009.0220","article-title":"Treatment of malignant pleural effusion: PleuRx catheter or talc pleurodesis? A cost-effectiveness analysis","volume":"13","author":"Olden","year":"2010","journal-title":"J. Palliat. Med."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1016\/S0893-6080(05)80023-1","article-title":"Stacked generalization","volume":"5","author":"Wolpert","year":"1992","journal-title":"Neural Netw."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"236","DOI":"10.1109\/TAFFC.2020.3047582","article-title":"A Multi-Modal Stacked Ensemble Model for Bipolar Disorder Classification Tree","volume":"14","author":"AbaeiKoupaei","year":"2023","journal-title":"IEEE Trans. Affect. Comput."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Aboneh, T., Rorissa, A., and Srinivasagan, R. (2022). Stacking-Based Ensemble Learning Method for Multi-Spectral Image Classification. Technologies, 10.","DOI":"10.3390\/technologies10010017"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"831","DOI":"10.1177\/00131644221117193","article-title":"Exploration of the Stacking Ensemble Machine Learning Algorithm for Cheating Detection in Large-Scale Assessment","volume":"83","author":"Zhou","year":"2023","journal-title":"Educ. Psychol. Meas."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"6675638","DOI":"10.1155\/2021\/6675638","article-title":"A Novel Stacking Heterogeneous Ensemble Model with Hybrid Wrapper-Based Feature Selection for Reservoir Productivity Predictions","volume":"2021","author":"Zhou","year":"2021","journal-title":"Complexity"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zheng, J.Y., Wang, M.H., Yao, T.C., Tang, Y.C., and Liu, H.J. (2023). Dynamic Mechanical Strength Prediction of BFRC Based on Stacking Ensemble Learning and Genetic Algorithm Optimization. Buildings, 13.","DOI":"10.3390\/buildings13051155"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"113692","DOI":"10.1109\/ACCESS.2021.3099795","article-title":"A Stacking Ensemble Prediction Model for the Occurrences of Major Adverse Cardiovascular Events in Patients with Acute Coronary Syndrome on Imbalanced Data","volume":"9","author":"Zheng","year":"2021","journal-title":"IEEE Access"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"E336","DOI":"10.3747\/co.22.2563","article-title":"Pleural fluid tumour markers in malignant pleural effusion with inconclusive cytologic results","volume":"22","author":"Antonangelo","year":"2015","journal-title":"Curr. Oncol."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"1610280","DOI":"10.3389\/pore.2022.1610280","article-title":"Diagnostic Value of Six Tumor Markers for Malignant Pleural Effusion in 1,230 Patients: A Single-Center Retrospective Study","volume":"28","author":"Fan","year":"2022","journal-title":"Pathol. Oncol. Res."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"283","DOI":"10.1007\/s10147-016-1073-y","article-title":"Diagnostic value of tumor markers for lung adenocarcinoma-associated malignant pleural effusion: A validation study and meta-analysis","volume":"22","author":"Feng","year":"2017","journal-title":"Int. J. Clin. Oncol."},{"key":"ref_26","first-page":"1001","article-title":"Diagnostic value of MUC1 and EpCAM mRNA as tumor markers in differentiating benign from malignant pleural effusion","volume":"107","author":"Sun","year":"2014","journal-title":"Int. J. Med."},{"key":"ref_27","unstructured":"(2023, August 03). Healthcare Security Administration of Hubei Province. No. 70, Available online: https:\/\/ybj.hubei.gov.cn\/zfxxgk\/zc\/gfwj\/201911\/t20191129_1545137.shtml."},{"key":"ref_28","unstructured":"(2023, August 03). Beijing Municipal Medical Insurance Bureau. No. 35, Available online: http:\/\/ybj.beijing.gov.cn\/zwgk\/2020_zcwj\/202112\/t20211231_2581064.html."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1080\/00365513.2020.1728784","article-title":"Differential diagnosis of tuberculous and malignant pleural effusions: Comparison of the Th1\/Th2 cytokine panel, tumor marker panel and chemistry panel","volume":"80","author":"Zhang","year":"2020","journal-title":"Scand. J. Clin. Lab. Inv."},{"key":"ref_30","first-page":"2899","article-title":"Effects of bevacizumab combined with oxaliplatin intrathoracic injection on tumor markers and survival rate in patients with malignant pleural effusion of lung cancer","volume":"13","author":"Chen","year":"2021","journal-title":"Am. J. Transl. Res."},{"key":"ref_31","first-page":"74","article-title":"Diagnostic value of combination measurement of tumor markers for differentiating malignant and pleural effusion","volume":"33","author":"Gu","year":"2011","journal-title":"J. Dalian Med. Univ."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"E37","DOI":"10.1002\/dc.24882","article-title":"Merkel cell carcinoma presenting as a malignant pleural effusion post-COVID-19 hospitalization: A case report and literature review","volume":"50","author":"Lanceta","year":"2022","journal-title":"Diagn. Cytopathol."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"302","DOI":"10.1016\/S0012-3692(15)34684-5","article-title":"Tumor Markers for Diagnosing Malignant Pleural Effusion?","volume":"121","author":"Paganuzzi","year":"2002","journal-title":"Chest"}],"container-title":["Diagnostics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2075-4418\/13\/19\/3136\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T21:01:47Z","timestamp":1760130107000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2075-4418\/13\/19\/3136"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,5]]},"references-count":33,"journal-issue":{"issue":"19","published-online":{"date-parts":[[2023,10]]}},"alternative-id":["diagnostics13193136"],"URL":"https:\/\/doi.org\/10.3390\/diagnostics13193136","relation":{},"ISSN":["2075-4418"],"issn-type":[{"value":"2075-4418","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,5]]}}}