{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,11,20]],"date-time":"2026-11-20T04:41:56Z","timestamp":1795149716882,"version":"4.10.3"},"reference-count":69,"publisher":"American Association for Cancer Research (AACR)","issue":"6","license":[{"start":{"date-parts":[[2023,6,29]],"date-time":"2023-06-29T00:00:00Z","timestamp":1687996800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100002879","name":"Canadian HIV Trials Network, Canadian Institutes of Health Research","doi-asserted-by":"crossref","award":["426366"],"award-info":[{"award-number":["426366"]}],"id":[{"id":"10.13039\/501100002879","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["aacrjournals.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,6,29]]},"abstract":"<jats:p>Artificial intelligence (AI) and machine learning (ML) are becoming critical in developing and deploying personalized medicine and targeted clinical trials. Recent advances in ML have enabled the integration of wider ranges of data including both medical records and imaging (radiomics). However, the development of prognostic models is complex as no modeling strategy is universally superior to others and validation of developed models requires large and diverse datasets to demonstrate that prognostic models developed (regardless of method) from one dataset are applicable to other datasets both internally and externally. Using a retrospective dataset of 2,552 patients from a single institution and a strict evaluation framework that included external validation on three external patient cohorts (873 patients), we crowdsourced the development of ML models to predict overall survival in head and neck cancer (HNC) using electronic medical records (EMR) and pretreatment radiological images. To assess the relative contributions of radiomics in predicting HNC prognosis, we compared 12 different models using imaging and\/or EMR data. The model with the highest accuracy used multitask learning on clinical data and tumor volume, achieving high prognostic accuracy for 2-year and lifetime survival prediction, outperforming models relying on clinical data only, engineered radiomics, or complex deep neural network architecture. However, when we attempted to extend the best performing models from this large training dataset to other institutions, we observed significant reductions in the performance of the model in those datasets, highlighting the importance of detailed population-based reporting for AI\/ML model utility and stronger validation frameworks. We have developed highly prognostic models for overall survival in HNC using EMRs and pretreatment radiological images based on a large, retrospective dataset of 2,552 patients from our institution.Diverse ML approaches were used by independent investigators. The model with the highest accuracy used multitask learning on clinical data and tumor volume.External validation of the top three performing models on three datasets (873 patients) with significant differences in the distributions of clinical and demographic variables demonstrated significant decreases in model performance.<\/jats:p>\n                  <jats:sec>\n                    <jats:title>Significance:<\/jats:title>\n                    <jats:p>ML combined with simple prognostic factors outperformed multiple advanced CT radiomics and deep learning methods. ML models provided diverse solutions for prognosis of patients with HNC but their prognostic value is affected by differences in patient populations and require extensive validation.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1158\/2767-9764.crc-22-0152","type":"journal-article","created":{"date-parts":[[2023,6,7]],"date-time":"2023-06-07T10:03:32Z","timestamp":1686132212000},"page":"1140-1151","update-policy":"https:\/\/doi.org\/10.1158\/crossmark_policy","source":"Crossref","is-referenced-by-count":52,"title":["Multi-institutional Prognostic Modeling in Head and Neck Cancer: Evaluating Impact and Generalizability of Deep Learning and Radiomics"],"prefix":"10.1158","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-1579-7887","authenticated-orcid":true,"given":"Michal","family":"Kazmierski","sequence":"first","affiliation":[{"name":"1Department of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada."},{"name":"2Princess Margaret Cancer Centre, Toronto, Ontario, Canada."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5887-9826","authenticated-orcid":true,"given":"Mattea","family":"Welch","sequence":"additional","affiliation":[{"name":"1Department of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada."},{"name":"2Princess Margaret Cancer Centre, Toronto, Ontario, Canada."},{"name":"3TECHNA Institute, Toronto, Ontario, Canada."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6257-2213","authenticated-orcid":true,"given":"Sejin","family":"Kim","sequence":"additional","affiliation":[{"name":"1Department of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada."},{"name":"2Princess Margaret Cancer Centre, Toronto, Ontario, Canada."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1371-1250","authenticated-orcid":true,"given":"Chris","family":"McIntosh","sequence":"additional","affiliation":[{"name":"1Department of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada."},{"name":"3TECHNA Institute, Toronto, Ontario, Canada."},{"name":"4Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, Ontario, Canada."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6658-6217","authenticated-orcid":true,"given":"Katrina","family":"Rey-McIntyre","sequence":"additional","affiliation":[{"name":"4Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, Ontario, Canada."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8072-4388","authenticated-orcid":true,"given":"Shao Hui","family":"Huang","sequence":"additional","affiliation":[{"name":"4Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, Ontario, Canada."},{"name":"5Department of Radiation Oncology, University of Toronto, Ontario, Canada."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3628-2367","authenticated-orcid":true,"given":"Tirth","family":"Patel","sequence":"additional","affiliation":[{"name":"3TECHNA Institute, Toronto, Ontario, Canada."},{"name":"4Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, Ontario, Canada."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7797-2893","authenticated-orcid":true,"given":"Tony","family":"Tadic","sequence":"additional","affiliation":[{"name":"4Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, Ontario, Canada."},{"name":"5Department of Radiation Oncology, University of Toronto, Ontario, Canada."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9620-6876","authenticated-orcid":true,"given":"Michael","family":"Milosevic","sequence":"additional","affiliation":[{"name":"3TECHNA Institute, Toronto, Ontario, Canada."},{"name":"4Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, Ontario, Canada."},{"name":"5Department of Radiation Oncology, University of Toronto, Ontario, Canada."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4344-6486","authenticated-orcid":true,"given":"Fei-Fei","family":"Liu","sequence":"additional","affiliation":[{"name":"4Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, Ontario, Canada."},{"name":"5Department of Radiation Oncology, University of Toronto, Ontario, Canada."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5106-5152","authenticated-orcid":true,"given":"Adam","family":"Ryczkowski","sequence":"additional","affiliation":[{"name":"6Department of Medical Physics, Greater Poland Cancer Centre, Poznan, Poland."},{"name":"7Department of Electroradiology, University of Medical Sciences, Poznan, Poland."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-7160-8150","authenticated-orcid":true,"given":"Joanna","family":"Kazmierska","sequence":"additional","affiliation":[{"name":"7Department of Electroradiology, University of Medical Sciences, Poznan, Poland."},{"name":"8Department of Radiotherapy II, Greater Poland Cancer Centre, Poznan, Poland."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0369-7432","authenticated-orcid":true,"given":"Zezhong","family":"Ye","sequence":"additional","affiliation":[{"name":"9Artificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, Massachusetts."},{"name":"10Department of Radiation Oncology, Dana-Farber Cancer Institute \/ Brigham and Women's Hosptial, Boston, Massachusetts."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4218-1693","authenticated-orcid":true,"given":"Deborah","family":"Plana","sequence":"additional","affiliation":[{"name":"9Artificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, Massachusetts."},{"name":"10Department of Radiation Oncology, Dana-Farber Cancer Institute \/ Brigham and Women's Hosptial, Boston, Massachusetts."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2122-2003","authenticated-orcid":true,"given":"Hugo J.W.L.","family":"Aerts","sequence":"additional","affiliation":[{"name":"9Artificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, Massachusetts."},{"name":"10Department of Radiation Oncology, Dana-Farber Cancer Institute \/ Brigham and Women's Hosptial, Boston, Massachusetts."},{"name":"11Radiology and Nuclear Medicine, CARIM and GROW, Maastricht University, Maastricht, the Netherlands."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4313-2754","authenticated-orcid":true,"given":"Benjamin H.","family":"Kann","sequence":"additional","affiliation":[{"name":"9Artificial Intelligence in Medicine (AIM) Program, Mass General Brigham, Harvard Medical School, Boston, Massachusetts."},{"name":"10Department of Radiation Oncology, Dana-Farber Cancer Institute \/ Brigham and Women's Hosptial, Boston, Massachusetts."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-8610-4908","authenticated-orcid":true,"given":"Scott V.","family":"Bratman","sequence":"additional","affiliation":[{"name":"1Department of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada."},{"name":"4Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, Ontario, Canada."},{"name":"5Department of Radiation Oncology, University of Toronto, Ontario, Canada."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6793-2863","authenticated-orcid":true,"given":"Andrew J.","family":"Hope","sequence":"additional","affiliation":[{"name":"4Radiation Medicine Program, Princess Margaret Cancer Centre, Toronto, Ontario, Canada."},{"name":"5Department of Radiation Oncology, University of Toronto, Ontario, Canada."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7684-0079","authenticated-orcid":true,"given":"Benjamin","family":"Haibe-Kains","sequence":"additional","affiliation":[{"name":"1Department of Medical Biophysics, University of Toronto, Toronto, Ontario, Canada."},{"name":"2Princess Margaret Cancer Centre, Toronto, Ontario, Canada."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1086","published-online":{"date-parts":[[2023,6,29]]},"reference":[{"key":"2023062914090141400_bib1","doi-asserted-by":"crossref","first-page":"448","DOI":"10.1001\/archinte.1971.00310150108015","article-title":"Computer-aided prognosis: II. Development of a prognostic algorithm","volume":"127","author":"Koss","year":"2022","journal-title":"Arch Intern Med"},{"key":"2023062914090141400_bib2","doi-asserted-by":"crossref","first-page":"954","DOI":"10.1038\/s41591-019-0447-x","article-title":"End-to-end lung cancer screening with three-dimensional deep learning on low-dose chest computed tomography","volume":"25","author":"Ardila","year":"2019","journal-title":"Nat Med"},{"key":"2023062914090141400_bib3","doi-asserted-by":"crossref","first-page":"170","DOI":"10.1186\/1471-2105-14-170","article-title":"Oral cancer prognosis based on clinicopathologic and genomic markers using a hybrid of feature selection and machine learning methods","volume":"14","author":"Chang","year":"2013","journal-title":"BMC Bioinformatics"},{"key":"2023062914090141400_bib4","doi-asserted-by":"crossref","first-page":"e228","DOI":"10.1002\/mp.13562","article-title":"Artificial intelligence-based clinical decision support in modern medical physics: selection, acceptance, commissioning, and quality assurance","volume":"47","author":"Mahadevaiah","year":"2020","journal-title":"Med Phys"},{"key":"2023062914090141400_bib5","doi-asserted-by":"crossref","first-page":"109","DOI":"10.1038\/nrclinonc.2013.244","article-title":"Rapid learning for precision oncology","volume":"11","author":"Shrager","year":"2014","journal-title":"Nat Rev Clin Oncol"},{"key":"2023062914090141400_bib6","doi-asserted-by":"crossref","first-page":"49","DOI":"10.1146\/annurev.pathol.4.110807.092158","article-title":"Molecular pathology of head and neck cancer: implications for diagnosis, prognosis, and treatment","volume":"4","author":"Pai","year":"2009","journal-title":"Annu Rev Pathol Mech Dis"},{"key":"2023062914090141400_bib7","doi-asserted-by":"crossref","first-page":"9","DOI":"10.1038\/nrc2982","article-title":"The molecular biology of head and neck cancer","volume":"11","author":"Leemans","year":"2011","journal-title":"Nat Rev Cancer"},{"key":"2023062914090141400_bib8","doi-asserted-by":"crossref","first-page":"1494","DOI":"10.1002\/ijc.28847","article-title":"Treatment de-escalation in HPV-positive oropharyngeal carcinoma: ongoing trials, critical issues and perspectives","volume":"136","author":"Mirghani","year":"2015","journal-title":"Int J Cancer"},{"key":"2023062914090141400_bib9","doi-asserted-by":"crossref","first-page":"543","DOI":"10.1200\/JCO.2012.44.0164","article-title":"Deintensification candidate subgroups in human papillomavirus\u2013related oropharyngeal cancer according to minimal risk of distant metastasis","volume":"31","author":"O'Sullivan","year":"2013","journal-title":"J Clin Oncol"},{"key":"2023062914090141400_bib10","doi-asserted-by":"crossref","first-page":"563","DOI":"10.1148\/radiol.2015151169","article-title":"Radiomics: images are more than pictures, they are data","volume":"278","author":"Gillies","year":"2019","journal-title":"Radiology"},{"key":"2023062914090141400_bib11","doi-asserted-by":"crossref","first-page":"436","DOI":"10.1038\/nature14539","article-title":"Deep learning","volume":"521","author":"LeCun","year":"2015","journal-title":"Nature"},{"key":"2023062914090141400_bib12","doi-asserted-by":"crossref","first-page":"371","DOI":"10.21037\/tcr.2016.07.18","article-title":"Radiomics in head and neck cancer: from exploration to application","volume":"5","author":"Wong","year":"2016","journal-title":"Transl Cancer Res"},{"key":"2023062914090141400_bib13","doi-asserted-by":"crossref","first-page":"e0222509","DOI":"10.1371\/journal.pone.0222509","article-title":"Radiomics features of the primary tumor fail to improve prediction of overall survival in large cohorts of CT- and PET-imaged head and neck cancer patients","volume":"14","author":"Ger","year":"2019","journal-title":"PLoS One"},{"key":"2023062914090141400_bib14","doi-asserted-by":"crossref","first-page":"2268","DOI":"10.1109\/JBHI.2019.2956354","article-title":"Multi-level multi-modality fusion radiomics: application to PET and CT imaging for prognostication of head and neck cancer","volume":"24","author":"Lv","year":"2020","journal-title":"IEEE J Biomed Health Inform"},{"key":"2023062914090141400_bib15","doi-asserted-by":"crossref","first-page":"10117","DOI":"10.1038\/s41598-017-10371-5","article-title":"Radiomics strategies for risk assessment of tumour failure in head-and-neck cancer","volume":"7","author":"Valli\u00e8res","year":"2017","journal-title":"Sci Rep"},{"key":"2023062914090141400_bib16","doi-asserted-by":"crossref","first-page":"2764","DOI":"10.1038\/s41598-019-39206-1","article-title":"Deep learning in head & neck cancer outcome prediction","volume":"9","author":"Diamant","year":"2019","journal-title":"Sci Rep"},{"key":"2023062914090141400_bib17","doi-asserted-by":"crossref","first-page":"131","DOI":"10.1186\/s13014-019-1339-4","article-title":"Predicting acute radiation induced xerostomia in head and neck Cancer using MR and CT Radiomics of parotid and submandibular glands","volume":"14","author":"Sheikh","year":"2019","journal-title":"Radiat Oncol"},{"key":"2023062914090141400_bib18","doi-asserted-by":"crossref","first-page":"12483","DOI":"10.1038\/s41598-019-48184-3","article-title":"Delta-radiomics features during radiotherapy improve the prediction of late xerostomia","volume":"9","author":"van Dijk","year":"2019","journal-title":"Sci Rep"},{"key":"2023062914090141400_bib19","doi-asserted-by":"crossref","first-page":"70","DOI":"10.1016\/j.ebiom.2019.06.034","article-title":"Development and validation of radiomic signatures of head and neck squamous cell carcinoma molecular features and subtypes","volume":"45","author":"Huang","year":"2019","journal-title":"EBioMedicine"},{"key":"2023062914090141400_bib20","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1200\/CCI.18.00073","article-title":"Imaging-genomic study of head and neck squamous cell carcinoma: associations between radiomic phenotypes and genomic mechanisms via integration of the cancer genome atlas and the cancer imaging archive","volume":"3","author":"Zhu","year":"2019","journal-title":"JCO Clin Cancer Inform"},{"key":"2023062914090141400_bib21","doi-asserted-by":"crossref","first-page":"e23421","DOI":"10.7554\/eLife.23421","article-title":"Defining the biological basis of radiomic phenotypes in lung cancer","volume":"6","author":"Grossmann","journal-title":"Elife"},{"key":"2023062914090141400_bib22","doi-asserted-by":"crossref","first-page":"349","DOI":"10.1016\/j.radonc.2018.03.033","article-title":"Tracking tumor biology with radiomics: a systematic review utilizing a radiomics quality score","volume":"127","author":"Sanduleanu","year":"2018","journal-title":"Radiother Oncol"},{"key":"2023062914090141400_bib23","doi-asserted-by":"crossref","first-page":"1074","DOI":"10.1016\/j.ijrobp.2018.08.032","article-title":"A deep look into the future of quantitative imaging in oncology: a statement of working principles and proposal for change","volume":"102","author":"Morin","year":"2018","journal-title":"Int J Radiat Oncol Biol Phys"},{"key":"2023062914090141400_bib24","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1109\/4235.585893","article-title":"No free lunch theorems for optimization","volume":"1","author":"Wolpert","year":"1997","journal-title":"IEEE Trans Evol Computat"},{"key":"2023062914090141400_bib25","doi-asserted-by":"crossref","first-page":"725","DOI":"10.1126\/science.359.6377.725","article-title":"Artificial intelligence faces reproducibility crisis","volume":"359","author":"Hutson","year":"2018","journal-title":"Science"},{"key":"2023062914090141400_bib26","doi-asserted-by":"crossref","first-page":"487","DOI":"10.1148\/radiol.2019192515","article-title":"Assessing radiology research on artificial intelligence: a brief guide for authors, reviewers, and readers\u2014from the radiology editorial board","volume":"294","author":"Bluemke","year":"2020","journal-title":"Radiology"},{"key":"2023062914090141400_bib27","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1148\/radiol.2020191145","article-title":"The image biomarker standardization initiative: standardized quantitative radiomics for high-throughput image-based phenotyping","volume":"295","author":"Zwanenburg","year":"2020","journal-title":"Radiology"},{"key":"2023062914090141400_bib28","doi-asserted-by":"crossref","first-page":"E14","DOI":"10.1038\/s41586-020-2766-y","article-title":"Transparency and reproducibility in artificial intelligence","volume":"586","author":"Haibe-Kains","year":"2020","journal-title":"Nature"},{"key":"2023062914090141400_bib29","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1016\/j.radonc.2018.10.027","article-title":"Vulnerabilities of radiomic signature development: the need for safeguards","volume":"130","author":"Welch","year":"2019","journal-title":"Radiother Oncol"},{"key":"2023062914090141400_bib30","doi-asserted-by":"crossref","first-page":"24","DOI":"10.1016\/j.ejmp.2020.02.010","article-title":"Machine learning helps identifying volume-confounding effects in radiomics","volume":"71","author":"Traverso","year":"2020","journal-title":"Physica Med"},{"key":"2023062914090141400_bib31","doi-asserted-by":"crossref","first-page":"e104","DOI":"10.1158\/0008-5472.CAN-17-0339","article-title":"Computational radiomics system to decode the radiographic phenotype","volume":"77","author":"van Griethuysen","year":"2017","journal-title":"Cancer Res"},{"key":"2023062914090141400_bib32","doi-asserted-by":"crossref","first-page":"1226","DOI":"10.1109\/TPAMI.2005.159","article-title":"Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy","volume":"27","author":"Peng","year":"2005","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"2023062914090141400_bib33","doi-asserted-by":"crossref","first-page":"861","DOI":"10.1016\/j.patrec.2005.10.010","article-title":"An introduction to ROC analysis","volume":"27","author":"Fawcett","year":"2006","journal-title":"Pattern Recognit Lett"},{"key":"2023062914090141400_bib34","doi-asserted-by":"crossref","first-page":"603","DOI":"10.1038\/nmeth.3945","article-title":"Classification evaluation","volume":"13","author":"Lever","year":"2016","journal-title":"Nat Methods"},{"key":"2023062914090141400_bib35","doi-asserted-by":"crossref","first-page":"4006","DOI":"10.1038\/ncomms5006","article-title":"Decoding tumour phenotype by noninvasive imaging using a quantitative radiomics approach","volume":"5","author":"Aerts","year":"2014","journal-title":"Nat Commun"},{"key":"2023062914090141400_bib36","doi-asserted-by":"crossref","first-page":"180173","DOI":"10.1038\/sdata.2018.173","article-title":"Imaging and clinical data archive for head and neck squamous cell carcinoma patients treated with radiotherapy","volume":"5","author":"Grossberg","year":"2018","journal-title":"Sci Data"},{"key":"2023062914090141400_bib37","doi-asserted-by":"crossref","first-page":"1045","DOI":"10.1007\/s10278-013-9622-7","article-title":"The cancer imaging archive (TCIA): maintaining and operating a public information repository","volume":"26","author":"Clark","year":"2013","journal-title":"J Digit Imaging"},{"key":"2023062914090141400_bib38","doi-asserted-by":"crossref","first-page":"e0118432","DOI":"10.1371\/journal.pone.0118432","article-title":"The precision-recall plot is more informative than the ROC plot when evaluating binary classifiers on imbalanced datasets","volume":"10","author":"Saito","year":"2015","journal-title":"PLoS One"},{"key":"2023062914090141400_bib39","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1002\/(SICI)1097-0258(19960229)15:4<361::AID-SIM168>3.0.CO;2-4","article-title":"Multivariable prognostic models: Issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors","volume":"15","author":"Harrell","year":"1996","journal-title":"Stat Med"},{"key":"2023062914090141400_bib40","article-title":"Densely connected convolutional networks","author":"Huang","year":"2016"},{"key":"2023062914090141400_bib41","doi-asserted-by":"crossref","first-page":"1342","DOI":"10.1038\/s41591-018-0107-6","article-title":"Clinically applicable deep learning for diagnosis and referral in retinal disease","volume":"24","author":"De Fauw","year":"2018","journal-title":"Nat Med"},{"key":"2023062914090141400_bib42","doi-asserted-by":"crossref","first-page":"i446","DOI":"10.1093\/bioinformatics\/btz342","article-title":"Deep learning with multimodal representation for pancancer prognosis prediction","volume":"35","author":"Cheerla","year":"2019","journal-title":"Bioinformatics"},{"key":"2023062914090141400_bib43","doi-asserted-by":"crossref","first-page":"960","DOI":"10.1002\/hed.24714","article-title":"Tumor volume as an independent predictive factor of worse survival in patients with oral cavity squamous cell carcinoma: tumor volume as predictive of survival in patients with oral cavity SCC","volume":"39","author":"Lin","year":"2017","journal-title":"Head Neck"},{"key":"2023062914090141400_bib44","first-page":"1845","article-title":"Learning patient-specific cancer survival distributions as a sequence of dependent regressors","volume-title":"Advances in neural information processing systems 24","author":"Yu","year":"2011"},{"key":"2023062914090141400_bib45","doi-asserted-by":"crossref","first-page":"274","DOI":"10.1038\/s42256-020-0173-6","article-title":"A shallow convolutional neural network predicts prognosis of lung cancer patients in multi-institutional computed tomography image datasets","volume":"2","author":"Mukherjee","year":"2020","journal-title":"Nat Mach Intell"},{"key":"2023062914090141400_bib46","article-title":"Deep neural networks for survival analysis based on a multi-task framework","author":"Fotso","year":"2018"},{"key":"2023062914090141400_bib47","article-title":"Fast and accurate deep network learning by exponential linear units (ELUs)","author":"Clevert","year":"2016"},{"key":"2023062914090141400_bib48","doi-asserted-by":"crossref","first-page":"e214732","DOI":"10.1001\/jamanetworkopen.2021.4732","article-title":"Comparison of clinical characteristics between clinical trial participants and nonparticipants using electronic health record data","volume":"4","author":"Rogers","year":"2021","journal-title":"JAMA Netw Open"},{"key":"2023062914090141400_bib49","doi-asserted-by":"crossref","first-page":"82","DOI":"10.1016\/S0140-6736(04)17670-8","article-title":"External validity of randomised controlled trials: \u201cTo whom do the results of this trial apply?","volume":"365","author":"Rothwell","year":"2005","journal-title":"Lancet"},{"key":"2023062914090141400_bib50","first-page":"e489","article-title":"The myth of generalisability in clinical research and machine learning in health care","volume-title":"Lancet Digital Health","author":"Futoma","year":"2020"},{"key":"2023062914090141400_bib51","doi-asserted-by":"crossref","DOI":"10.1007\/978-3-030-58558-7_29","article-title":"Big Transfer (BiT): general visual representation learning","author":"Kolesnikov","year":"2020"},{"key":"2023062914090141400_bib52","first-page":"843","article-title":"Revisiting unreasonable effectiveness of data in deep learning era","author":"Sun","year":"2017"},{"key":"2023062914090141400_bib53","doi-asserted-by":"crossref","first-page":"e1001779","DOI":"10.1371\/journal.pmed.1001779","article-title":"UK Biobank: an open access resource for identifying the causes of a wide range of complex diseases of middle and old age","volume":"12","author":"Sudlow","year":"2015","journal-title":"PLoS Med"},{"key":"2023062914090141400_bib54","doi-asserted-by":"crossref","first-page":"1113","DOI":"10.1038\/ng.2764","article-title":"The Cancer Genome Atlas pan-cancer analysis project","volume":"45","author":"Cancer Genome Atlas Research Network","year":"2013","journal-title":"Nat Genet"},{"key":"2023062914090141400_bib55","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1016\/j.compmedimag.2015.04.006","article-title":"Preliminary investigation into sources of uncertainty in quantitative imaging features","volume":"44","author":"Fave","year":"2015","journal-title":"Comput Med Imaging Graph"},{"key":"2023062914090141400_bib56","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1007\/s00259-016-3427-0","article-title":"Characterization of PET\/CT images using texture analysis: the past, the present\u2026 any future?","volume":"44","author":"Hatt","year":"2019","journal-title":"Eur J Nucl Med Mol Imaging"},{"key":"2023062914090141400_bib57","doi-asserted-by":"crossref","first-page":"554","DOI":"10.1038\/s42256-020-00239-1","article-title":"Improving the quality of machine learning in health applications and clinical research","volume":"2","author":"Mateen","year":"2020","journal-title":"Nat Mach Intell"},{"key":"2023062914090141400_bib58","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1016\/j.radonc.2020.09.054","article-title":"From multisource data to clinical decision aids in radiation oncology: the need for a clinical data science community","volume":"153","author":"Kazmierska","year":"2020","journal-title":"Radiother Oncol"},{"key":"2023062914090141400_bib59","first-page":"1","article-title":"Ensemble methods in machine learning","author":"Dietterich","year":"2000"},{"key":"2023062914090141400_bib60","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1016\/S0893-6080(05)80023-1","article-title":"Stacked generalization","volume":"5","author":"Wolpert","year":"1992","journal-title":"Neural Netw"},{"key":"2023062914090141400_bib61","doi-asserted-by":"crossref","first-page":"857","DOI":"10.1002\/hed.25336","article-title":"Impact of American Joint Committee on Cancer Eighth Edition clinical stage and smoking history on oncologic outcomes in human papillomavirus-associated oropharyngeal squamous cell carcinoma","volume":"41","author":"Hawkins","year":"2019","journal-title":"Head Neck"},{"key":"2023062914090141400_bib62","first-page":"6047","article-title":"Impact of tobacco smoking on radiotherapy outcomes in 1875 HPV-positive oropharynx cancer patients","volume":"37","author":"Lassen","year":"2019","journal-title":"J Clin Orthod"},{"key":"2023062914090141400_bib63","doi-asserted-by":"crossref","first-page":"4","DOI":"10.1093\/oncolo\/oyab010","article-title":"Unfinished business in classifying HPV-Positive oropharyngeal carcinoma: identifying the bad apples in a good staging barrel","volume":"27","author":"Huang","year":"2022","journal-title":"Oncologist"},{"key":"2023062914090141400_bib64","article-title":"Adam: a method for stochastic optimization","author":"Kingma","year":"2014"},{"key":"2023062914090141400_bib65","doi-asserted-by":"crossref","first-page":"R18","DOI":"10.1186\/gb-2010-11-2-r18","article-title":"A fuzzy gene expression-based computational approach improves breast cancer prognostication","volume":"11","author":"Haibe-Kains","year":"2010","journal-title":"Genome Biol"},{"key":"2023062914090141400_bib66","first-page":"2825","article-title":"Scikit-learn: machine learning in python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J Mach Learn Res"},{"key":"2023062914090141400_bib67","first-page":"448","article-title":"Batch normalization: accelerating deep network training by reducing internal covariate shift","volume-title":"Proceedings of the 32nd International Conference on Machine Learning","author":"Ioffe","year":"2015"},{"key":"2023062914090141400_bib68","article-title":"Self-normalizing neural networks","author":"Klambauer","year":"2017"},{"key":"2023062914090141400_bib69","article-title":"Very deep convolutional networks for large-scale image recognition","author":"Simonyan","year":"2015"}],"container-title":["Cancer Research Communications"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/aacrjournals.org\/cancerrescommun\/article-pdf\/doi\/10.1158\/2767-9764.CRC-22-0152\/3339476\/crc-22-0152.pdf","content-type":"application\/pdf","content-version":"am","intended-application":"syndication"},{"URL":"https:\/\/aacrjournals.org\/cancerrescommun\/article-pdf\/3\/6\/1140\/3344154\/crc-22-0152.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/aacrjournals.org\/cancerrescommun\/article-pdf\/3\/6\/1140\/3344154\/crc-22-0152.pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,29]],"date-time":"2023-06-29T10:10:01Z","timestamp":1688033401000},"score":1,"resource":{"primary":{"URL":"https:\/\/aacrjournals.org\/cancerrescommun\/article\/3\/6\/1140\/727552\/Multi-institutional-Prognostic-Modeling-in-Head"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,29]]},"references-count":69,"journal-issue":{"issue":"6","published-print":{"date-parts":[[2023,6,29]]}},"URL":"https:\/\/doi.org\/10.1158\/2767-9764.crc-22-0152","relation":{"is-supplemented-by":[{"id-type":"doi","id":"10.1158\/2767-9764.23600396.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23312528","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600408.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600399.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23312540","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600381","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600393","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23312540.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600405.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600402.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23312537","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600381.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600402","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23312537.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600384.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23312528.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600384","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600390.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23312531.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23312534.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600393.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600387.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23312534","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600405","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600396","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600408","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23312531","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600387","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600399","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600390","asserted-by":"object"}],"has-preprint":[{"id-type":"doi","id":"10.1158\/2767-9764.23600387.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600387","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.c.6684845.v2","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600381.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600399","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23312531.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600384.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23312537","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600408","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23312537.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23312540.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23312534.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.c.6684845.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600396","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600384","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600405","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23312540","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600402.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23312528","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600393.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600405.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600396.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600393","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600381","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600408.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23312531","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600399.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600402","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600390.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23312528.v1","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23600390","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.c.6684845","asserted-by":"object"},{"id-type":"doi","id":"10.1158\/2767-9764.23312534","asserted-by":"object"}]},"ISSN":["2767-9764"],"issn-type":[{"value":"2767-9764","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,6,29]]}}}