{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,30]],"date-time":"2026-05-30T00:58:10Z","timestamp":1780102690535,"version":"3.54.0"},"reference-count":39,"publisher":"Springer Science and Business Media LLC","issue":"1","license":[{"start":{"date-parts":[[2021,7,8]],"date-time":"2021-07-08T00:00:00Z","timestamp":1625702400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,7,8]],"date-time":"2021-07-08T00:00:00Z","timestamp":1625702400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["BMC Med Inform Decis Mak"],"published-print":{"date-parts":[[2021,12]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:sec>\n                <jats:title>Background<\/jats:title>\n                <jats:p>Machine learning (ML) can be an effective tool to extract information from attribute-rich molecular datasets for the generation of molecular diagnostic tests. However, the way in which the resulting scores or classifications are produced from the input data may not be transparent. Algorithmic explainability or interpretability has become a focus of ML research. Shapley values, first introduced in game theory, can provide explanations of the result generated from a specific set of input data by a complex ML algorithm.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Methods<\/jats:title>\n                <jats:p>For a multivariate molecular diagnostic test in clinical use (the VeriStrat\u00ae test), we calculate and discuss the interpretation of exact Shapley values. We also employ some standard approximation techniques for Shapley value computation (local interpretable model-agnostic explanation (<jats:italic>LIME<\/jats:italic>) and Shapley Additive Explanations (<jats:italic>SHAP<\/jats:italic>) based methods) and compare the results with exact Shapley values.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Results<\/jats:title>\n                <jats:p>Exact Shapley values calculated for data collected from a cohort of 256 patients showed that the relative importance of attributes for test classification varied by sample. While all eight features used in the VeriStrat\u00ae test contributed equally to classification for some samples, other samples showed more complex patterns of attribute importance for classification generation. Exact Shapley values and Shapley-based interaction metrics were able to provide interpretable classification explanations at the sample or patient level, while patient subgroups could be defined by comparing Shapley value profiles between patients. <jats:italic>LIME<\/jats:italic> and <jats:italic>SHAP<\/jats:italic> approximation approaches, even those seeking to include correlations between attributes, produced results that were quantitatively and, in some cases qualitatively, different from the exact Shapley values.<\/jats:p>\n              <\/jats:sec><jats:sec>\n                <jats:title>Conclusions<\/jats:title>\n                <jats:p>Shapley values can be used to determine the relative importance of input attributes to the result generated by a multivariate molecular diagnostic test for an individual sample or patient. Patient subgroups defined by Shapley value profiles may motivate translational research. However, correlations inherent in molecular data and the typically small ML training sets available for molecular diagnostic test development may cause some approximation methods to produce approximate Shapley values that differ both qualitatively and quantitatively from exact Shapley values. Hence, caution is advised when using approximate methods to evaluate Shapley explanations of the results of molecular diagnostic tests.<\/jats:p>\n              <\/jats:sec>","DOI":"10.1186\/s12911-021-01569-9","type":"journal-article","created":{"date-parts":[[2021,7,8]],"date-time":"2021-07-08T10:03:12Z","timestamp":1625738592000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":32,"title":["Explaining multivariate molecular diagnostic tests via Shapley values"],"prefix":"10.1186","volume":"21","author":[{"given":"Joanna","family":"Roder","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Laura","family":"Maguire","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"suffix":"III","given":"Robert","family":"Georgantas","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Heinrich","family":"Roder","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2021,7,8]]},"reference":[{"key":"1569_CR1","doi-asserted-by":"crossref","unstructured":"Molnar C, Casalicchio G, Bischl B. Interpretable machine learning\u2014a brief history, state-of-the-art and challenges. 2020. arXiv:2010.09337 [stat.ML]","DOI":"10.1007\/978-3-030-65965-3_28"},{"issue":"6464","key":"1569_CR2","doi-asserted-by":"publisher","first-page":"447","DOI":"10.1126\/science.aax2342","volume":"366","author":"Z Obermeyer","year":"2019","unstructured":"Obermeyer Z, Powers B, Vogeli C, Mullainathan S. Dissecting racial bias in an algorithm used to manage the health of populations. Science. 2019;366(6464):447\u201353. https:\/\/doi.org\/10.1126\/science.aax2342.","journal-title":"Science"},{"key":"1569_CR3","first-page":"671","volume":"104","author":"S Barocas","year":"2016","unstructured":"Barocas S, Selbst AD. Big data\u2019s disparate impact. Calif Law Rev. 2016;104:671.","journal-title":"Calif Law Rev"},{"key":"1569_CR4","unstructured":"European Union. Regulation (EU) 2016\/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95\/46\/EC (General Data Protection Regulation). Official Journal of the European Union 59. 2016"},{"key":"1569_CR5","doi-asserted-by":"publisher","unstructured":"Ribeiro MT, Singh S, Guestrin C. \u201cWhy should I trust you?\u201d explaining the predictions of any classifier. In: Proceedings of the 2016 conference of the North American Chapter of the Association for Computational Linguistics (Demonstrations). 2016;96\u2013101. https:\/\/doi.org\/10.18653\/v1\/N16-3020.","DOI":"10.18653\/v1\/N16-3020"},{"issue":"28","key":"1569_CR6","first-page":"307","volume":"2","author":"LS Shapley","year":"1953","unstructured":"Shapley LS. A value for n-person games. Contrib Theory Games. 1953;2(28):307\u201317.","journal-title":"Contrib Theory Games"},{"key":"1569_CR7","first-page":"325","volume-title":"Contributions to the theory of games IV","author":"JC Harsanyi","year":"1959","unstructured":"Harsanyi JC. A bargaining model for cooperative n-person games. In: Tucker AW, Luce RD, editors. Contributions to the theory of games IV. Princeton: Princeton University Press; 1959. p. 325\u201355."},{"issue":"10","key":"1569_CR8","doi-asserted-by":"publisher","first-page":"886","DOI":"10.1016\/j.datak.2009.01.004","volume":"68","author":"E \u0160trumbelj","year":"2009","unstructured":"\u0160trumbelj E, Kononenko I, \u0160ikonja MR. Explaining instance classifications with interactions of subsets of feature values. Data Knowl Eng. 2009;68(10):886\u2013904. https:\/\/doi.org\/10.1016\/j.datak.2009.01.004.","journal-title":"Data Knowl Eng"},{"key":"1569_CR9","first-page":"1","volume":"11","author":"E \u0160trumbelj","year":"2010","unstructured":"\u0160trumbelj E, Kononenko I. An efficient explanation of individual classifications using game theory. J Mach Learn Res. 2010;11:1\u201318.","journal-title":"J Mach Learn Res"},{"key":"1569_CR10","doi-asserted-by":"publisher","first-page":"647","DOI":"10.1007\/s10115-013-0679-x","volume":"41","author":"E \u0160trumbelj","year":"2014","unstructured":"\u0160trumbelj E, Kononenko I. Explaining prediction models and individual predictions with feature contributions. Knowl Inf Syst. 2014;41:647\u201365. https:\/\/doi.org\/10.1007\/s10115-013-0679-x.","journal-title":"Knowl Inf Syst"},{"key":"1569_CR11","doi-asserted-by":"publisher","first-page":"65","DOI":"10.1007\/BF01769885","volume":"14","author":"HP Young","year":"1985","unstructured":"Young HP. Monotonic solutions of cooperative games. Int J Game Theory. 1985;14:65\u201372.","journal-title":"Int J Game Theory"},{"key":"1569_CR12","doi-asserted-by":"crossref","unstructured":"Aas K, Jullum M, L\u00f8land A. Explaining individual predictions when features are dependent: more accurate approximations to Shapley values. 2020. arXiv:1903.10464v3 [stat.ML]","DOI":"10.1016\/j.artint.2021.103502"},{"key":"1569_CR13","unstructured":"Lundberg SM, Lee S. A Unified Approach to Interpreting Model Predictions. 31st Conference on Neural Information Processing Systems. 2017:4768\u20134777."},{"key":"1569_CR14","unstructured":"Lundberg SM, Erion GG, Lee S. Consistent individualized features attribution for tree ensembles. 2018. arXiv:1802.03888v3 [cs.LG]"},{"key":"1569_CR15","unstructured":"Jia R, Dao D, Wang B, Hubis FA, Hynes N, Gurel NM, Li B, Zhang C, Song D, Spanos C. Towards efficient data valuation based on the Shapley value. In: Proceedings of machine learning research. 2019;1167\u20131176"},{"key":"1569_CR16","unstructured":"Dhamdhere K, Agarwal A, Sundararajan M. The Shapley Taylor interaction index. 2020. arXiv:1902.05622v2 [cs.GT]"},{"issue":"11","key":"1569_CR17","doi-asserted-by":"publisher","first-page":"838","DOI":"10.1093\/jnci\/djk195","volume":"99","author":"F Taguchi","year":"2007","unstructured":"Taguchi F, Solomon B, Gregorc V, Roder H, Gray R, Kasahara K, Nishio M, Brahmer J, Spreafico A, Ludovini V, Massion PP, Dziadziuszko R, Schiller J, Grigorieva J, Tsypin M, Hunsucker SW, Caprioli R, Duncan MW, Hirsch FR, Bunn PA Jr, Carbone DP. Mass spectrometry to classify non-small-cell lung cancer patients for clinical outcome after treatment with epidermal growth factor receptor tyrosine kinase inhibitors: a multicohort cross-institutional study. J Natl Cancer Inst. 2007;99(11):838\u201346.","journal-title":"J Natl Cancer Inst"},{"issue":"7","key":"1569_CR18","doi-asserted-by":"publisher","first-page":"713","DOI":"10.1016\/S1470-2045(14)70162-7","volume":"15","author":"V Gregorc","year":"2014","unstructured":"Gregorc V, Novello S, Lazzari C, Barni S, Aieta M, Mencoboni M, Grossi F, De Pas T, de Marinis F, Bearz A, Floriani I, Torri V, Bulotta A, Cattaneo A, Grigorieva J, Tsypin M, Roder J, Doglioni C, Levra MG, Petrelli F, Foti S, Vigan\u00f2 M, Bachi A, Roder H. Predictive value of a proteomic signature in patients with non-small-cell lung cancer treated with second-line erlotinib or chemotherapy (PROSE): a biomarker-stratified, randomised phase 3 trial. Lancet Oncol. 2014;15(7):713\u201321. https:\/\/doi.org\/10.1016\/S1470-2045(14)70162-7.","journal-title":"Lancet Oncol"},{"issue":"2","key":"1569_CR19","doi-asserted-by":"publisher","first-page":"358","DOI":"10.1158\/1055-9965.EPI-09-0937","volume":"19","author":"CH Chung","year":"2010","unstructured":"Chung CH, Seeley EH, Roder H, Grigorieva J, Tsypin M, Roder J, Burtness BA, Argiris A, Forastiere AA, Gilbert J, Murphy B, Caprioli RM, Carbone DP, Cohen EEW. Detection of tumor epidermal growth factor receptor pathway dependence by serum mass spectrometry in cancer patients. Cancer Epidemiol Biomarkers Prev. 2010;19(2):358\u201365. https:\/\/doi.org\/10.1158\/1055-9965.EPI-09-0937.","journal-title":"Cancer Epidemiol Biomarkers Prev"},{"issue":"3","key":"1569_CR20","doi-asserted-by":"publisher","first-page":"572","DOI":"10.1002\/cam4.208","volume":"3","author":"DB Cardin","year":"2014","unstructured":"Cardin DB, Goff L, Li C, Shyr Y, Winkler C, de Vore R, Schlabach L, Holloway M, McClanahan P, Meyer K, Grigorieva J, Berlin J, Chan E. Phase II trial of sorafenib and erlotinib in advanced pancreatic cancer. Cancer Med. 2014;3(3):572\u20139. https:\/\/doi.org\/10.1002\/cam4.208.","journal-title":"Cancer Med"},{"issue":"9","key":"1569_CR21","doi-asserted-by":"publisher","first-page":"1497","DOI":"10.1080\/03007995.2020.1790346","volume":"36","author":"TA Leal","year":"2020","unstructured":"Leal TA, Argento AC, Bhadra K, Hogarth DK, Grigorieva J, Hartfield RM, McDonald RC, Bonomi PD. Prognostic performance of proteomic testing in advanced non-small cell lung cancer: a systematic literature review and meta-analysis. Curr Med Res Opin. 2020;36(9):1497\u2013505. https:\/\/doi.org\/10.1080\/03007995.2020.1790346.","journal-title":"Curr Med Res Opin"},{"key":"1569_CR22","unstructured":"Roder H, Tsypin M, Grigorieva J. Biodesix, Inc. Method and system for determining whether a drug will be effective on a patient with a disease. US Patent. US7736905B2. 2010."},{"key":"1569_CR23","doi-asserted-by":"publisher","first-page":"91","DOI":"10.1016\/j.jproy.2012.06.022","volume":"76","author":"E Milan","year":"2012","unstructured":"Milan E, Lazzari C, Anand S, Floriani I, Torri V, Sorlini C, Gregorc V, Bachi A. SAA1 is over-expressed in plasma of non-small cell lung cancer patients with poor outcome after treatment with epidermal growth factor receptor tyrosine-kinase inhibitors. J Proteomics. 2012;76:91\u2013101. https:\/\/doi.org\/10.1016\/j.jproy.2012.06.022.","journal-title":"J Proteomics"},{"issue":"1\u20133","key":"1569_CR24","doi-asserted-by":"publisher","first-page":"166","DOI":"10.1016\/s0014-5793(03)00097-8","volume":"537","author":"UA Kiernan","year":"2003","unstructured":"Kiernan UA, Tubbs KA, Nedelkov D, Niederkofler EE, Nelson RW. Detection of novel truncated forms of human serum amyloid A protein in human plasma. FEBS Lett. 2003;537(1\u20133):166\u201370. https:\/\/doi.org\/10.1016\/s0014-5793(03)00097-8.","journal-title":"FEBS Lett"},{"issue":"1","key":"1569_CR25","doi-asserted-by":"publisher","first-page":"310","DOI":"10.1186\/s12885-018-4193-0","volume":"18","author":"MJ Fidler","year":"2018","unstructured":"Fidler MJ, Fhied CL, Roder J, Basu S, Sayidine S, Fughhi I, Pool M, Batus M, Bonomi P, Borgia JA. The serum-based VeriStrat\u00ae test is associated with proinflammatory reactants and clinical outcome in non-small cell lung cancer patients. BMC Cancer. 2018;18(1):310. https:\/\/doi.org\/10.1186\/s12885-018-4193-0.","journal-title":"BMC Cancer"},{"key":"1569_CR26","unstructured":"Kumar IE, Venkatasubramanian S, Scheidegger C, Friedler S. In: Proceedings of the 37th international conference on machine learning, PMLR. 2020;119:5491\u20135500."},{"key":"1569_CR27","doi-asserted-by":"publisher","first-page":"389","DOI":"10.1023\/A:1012487302797","volume":"46","author":"I Guyon","year":"2002","unstructured":"Guyon I, Weston J, Barnhill S, Vapnik V. Gene selection for cancer classification using support vector machines. Mach Learn. 2002;46:389\u2013422.","journal-title":"Mach Learn"},{"key":"1569_CR28","first-page":"2579","volume":"9","author":"L van der Maaten","year":"2008","unstructured":"van der Maaten L, Hinton G. Visualizing data using t-SNE. J Mach Learn Res. 2008;9:2579\u2013605.","journal-title":"J Mach Learn Res"},{"key":"1569_CR29","first-page":"177","volume-title":"Pattern classification","author":"RO Duda","year":"2001","unstructured":"Duda RO, Hart PE, Stork DG. Pattern classification. 2nd ed. New York: Wiley; 2001. p. 177\u201384.","edition":"2"},{"issue":"11","key":"1569_CR30","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0239172","volume":"15","author":"D Ferrari","year":"2020","unstructured":"Ferrari D, Milic J, Tonelli R, Ghinelli F, Meschiari M, Volpi S, et al. Machine learning in predicting respiratory failure in patients with COVID-19 pneumonia\u2014challenges, strengths, and opportunities in a global health emergency. PLoS ONE. 2020;15(11): e0239172. https:\/\/doi.org\/10.1371\/journal.pone.0239172.","journal-title":"PLoS ONE"},{"issue":"11","key":"1569_CR31","doi-asserted-by":"publisher","DOI":"10.2196\/24018","volume":"22","author":"A Vaid","year":"2020","unstructured":"Vaid A, Somani S, Russak AJ, de Freitas JK, Chaudry FF, Paranjpe I, et al. Machine learning to predict mortality and critical events in a cohort of patients with COVID-19 in New York City: model development and validation. J Med Internet Res. 2020;22(11): e24018. https:\/\/doi.org\/10.2196\/24018.","journal-title":"J Med Internet Res"},{"key":"1569_CR32","doi-asserted-by":"publisher","first-page":"57","DOI":"10.3390\/cancers13010057","volume":"13","author":"P Giraud","year":"2021","unstructured":"Giraud P, Nicolas E, Boisselier P, Alfonsi M, Rives M, et al. Interpretable machine learning model for locoregional relapse prediction in oropharyngeal cancers. Cancers. 2021;13:57.","journal-title":"Cancers"},{"key":"1569_CR33","doi-asserted-by":"publisher","first-page":"4565","DOI":"10.1038\/s41598-021-83922-6","volume":"11","author":"A Paola Carrieri","year":"2021","unstructured":"Paola Carrieri A, Haiminen N, Sean Maudsley-Barton S, et al. Explainable AI reveals changes in skin microbiome composition linked to phenotypic differences. Nat Sci Rep. 2021;11:4565. https:\/\/doi.org\/10.1038\/s41598-021-83922-6.","journal-title":"Nat Sci Rep"},{"issue":"530","key":"1569_CR34","doi-asserted-by":"publisher","first-page":"636","DOI":"10.1080\/01621459.2020.1762613","volume":"115","author":"B Efron","year":"2020","unstructured":"Efron B. Prediction, estimation, and attribution. J Am Stat Assoc. 2020;115(530):636\u201355. https:\/\/doi.org\/10.1080\/01621459.2020.1762613.","journal-title":"J Am Stat Assoc"},{"key":"1569_CR35","doi-asserted-by":"publisher","unstructured":"Amann J, Blasimme A, Vayena E, Frey D, Madai VI, on behalf of the Precise4Q consortium. Explainability for artificial intelligence in healthcare: a multidisciplinary perspective. BMC Med Inform Decis Mak. 2020;20:310. https:\/\/doi.org\/10.1186\/s12911-020-01332-6.","DOI":"10.1186\/s12911-020-01332-6"},{"issue":"1","key":"1569_CR36","doi-asserted-by":"publisher","first-page":"325","DOI":"10.1186\/s12859-019-2922-2","volume":"20","author":"J Roder","year":"2019","unstructured":"Roder J, Oliveira C, Net L, Tsypin M, Linstid B, Roder H. A dropout-regularized classifier development approach optimized for precision medicine test discovery from omics data. BMC Bioinf. 2019;20(1):325. https:\/\/doi.org\/10.1186\/s12859-019-2922-2.","journal-title":"BMC Bioinf"},{"issue":"1","key":"1569_CR37","doi-asserted-by":"publisher","first-page":"79","DOI":"10.1158\/2326=6066.CIR-17-0412","volume":"6","author":"JS Weber","year":"2018","unstructured":"Weber JS, Sznol M, Sullivan RJ, Blackmon S, Boland G, Kluger HM, et al. A serum protein signature associated with outcome after anti-PD-1 therapy in metastatic melanoma. Cancer Immunol Res. 2018;6(1):79\u201386. https:\/\/doi.org\/10.1158\/2326=6066.CIR-17-0412.","journal-title":"Cancer Immunol Res"},{"issue":"19","key":"1569_CR38","doi-asserted-by":"publisher","first-page":"5188","DOI":"10.1158\/1078-0432.CCR-20-0538","volume":"26","author":"M Muller","year":"2020","unstructured":"Muller M, Hummelink K, Hurkmans DP, Niemeijer AN, Monkhorst K, Roder J, et al. A serum protein classifier identifying patients with advanced non-small cell lung cancer who derive clinical benefit from treatment with immune checkpoint inhibitors. Clin Cancer Res. 2020;26(19):5188\u201397. https:\/\/doi.org\/10.1158\/1078-0432.CCR-20-0538.","journal-title":"Clin Cancer Res"},{"issue":"9","key":"1569_CR39","doi-asserted-by":"publisher","first-page":"2519","DOI":"10.3390\/cancers12092519","volume":"12","author":"S Kasimir-Bauer","year":"2020","unstructured":"Kasimir-Bauer S, Roder J, Obermayr E, Mahner S, Vergote I, Loverix L, et al. Definition and independent validation of a proteomic-classifier in ovarian cancer. Cancers (Basel). 2020;12(9):2519. https:\/\/doi.org\/10.3390\/cancers12092519.","journal-title":"Cancers (Basel)"}],"container-title":["BMC Medical Informatics and Decision Making"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-021-01569-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1186\/s12911-021-01569-9\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1186\/s12911-021-01569-9.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,7,8]],"date-time":"2021-07-08T10:08:29Z","timestamp":1625738909000},"score":1,"resource":{"primary":{"URL":"https:\/\/bmcmedinformdecismak.biomedcentral.com\/articles\/10.1186\/s12911-021-01569-9"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,7,8]]},"references-count":39,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2021,12]]}},"alternative-id":["1569"],"URL":"https:\/\/doi.org\/10.1186\/s12911-021-01569-9","relation":{},"ISSN":["1472-6947"],"issn-type":[{"value":"1472-6947","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,7,8]]},"assertion":[{"value":"29 March 2021","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"29 June 2021","order":2,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"8 July 2021","order":3,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The study was approved by institutional review boards and independent ethics committees at each participating site (San Raffaele Scientific Institute, Milan, Italy; University of Turin, AOU San Luigi Orbassano, Turin, Italy; Azienda Opsedaliera Treviglio, Treviglio, Italy; IRCCS Centro di Riferimento Oncologico di Basilicata, Rionero in Vulture, Italy; Villa Scassi Hospital, Genoa, Italy; Azienda Ospedaliera San Martino University, Istituto Nazionale per la Ricerca sul Cancro, Genoa, Italy; European Institute of Oncology, Milan, Italy; San Camillo High Specialization Hospital, Rome, Italy; National Cancer Institute of Aviano, Aviano, Italy; Fondazione Istituto San Raffaele di G. Giglio di Cefalu, Cefalu, Italy; Azienda \u201cComplesso Ospedaliero San Fillippo Neri\u201d, Rome, Italy; Azienda Ospedaliera Spedali Civili di Brescia, Brescia, Italy; Azienda Ospedaliera Ospedale S. Anna, Como, Itlay; Fondazione IRCCS Istituto Nazionale dei Tumori, Milan, Italy). All patients provided written informed consent.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Ethics approval and consent to participate"}},{"value":"Not applicable.","order":3,"name":"Ethics","group":{"name":"EthicsHeading","label":"Consent for publication"}},{"value":"HR and JR are inventors on patents concerning the VeriStrat test, all assigned to Biodesix, Inc. All authors are current employees of Biodesix, Inc. and have stock options and\/or own shares in Biodesix, Inc.","order":4,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}],"article-number":"211"}}