{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,7]],"date-time":"2026-10-07T01:36:24Z","timestamp":1791336984406,"version":"4.3.1"},"reference-count":31,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2022,3,29]],"date-time":"2022-03-29T00:00:00Z","timestamp":1648512000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Cancers"],"abstract":"<jats:p>The histological distinction of lung neuroendocrine carcinoma, including small cell lung carcinoma (SCLC), large cell neuroendocrine carcinoma (LCNEC) and atypical carcinoid (AC), can be challenging in some cases, while bearing prognostic and therapeutic significance. To assist pathologists with the differentiation of histologic subtyping, we applied a deep learning classifier equipped with a convolutional neural network (CNN) to recognize lung neuroendocrine neoplasms. Slides of primary lung SCLC, LCNEC and AC were obtained from the Laboratory of Clinical and Experimental Pathology (University Hospital Nice, France). Three thoracic pathologists blindly established gold standard diagnoses. The HALO-AI module (Indica Labs, UK) trained with 18,752 image tiles extracted from 60 slides (SCLC = 20, LCNEC = 20, AC = 20 cases) was then tested on 90 slides (SCLC = 26, LCNEC = 22, AC = 13 and combined SCLC with LCNEC = 4 cases; NSCLC = 25 cases) by F1-score and accuracy. A HALO-AI correct area distribution (AD) cutoff of 50% or more was required to credit the CNN with the correct diagnosis. The tumor maps were false colored and displayed side by side to original hematoxylin and eosin slides with superimposed pathologist annotations. The trained HALO-AI yielded a mean F1-score of 0.99 (95% CI, 0.939\u20130.999) on the testing set. Our CNN model, providing further larger validation, has the potential to work side by side with the pathologist to accurately differentiate between the different lung neuroendocrine carcinoma in challenging cases.<\/jats:p>","DOI":"10.3390\/cancers14071740","type":"journal-article","created":{"date-parts":[[2022,3,29]],"date-time":"2022-03-29T21:40:14Z","timestamp":1648590014000},"page":"1740","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":24,"title":["Deep Learning Facilitates Distinguishing Histologic Subtypes of Pulmonary Neuroendocrine Tumors on Digital Whole-Slide Images"],"prefix":"10.3390","volume":"14","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-2014-1236","authenticated-orcid":false,"given":"Marius","family":"Ili\u00e9","sequence":"first","affiliation":[{"name":"Laboratory of Clinical and Experimental Pathology, Centre Hospitalier Universitaire de Nice, FHU OncoAge, Universit\u00e9 C\u00f4te d\u2019Azur, 06000 Nice, France"},{"name":"Hospital-Related Biobank (BB-0033-00025), Centre Hospitalier Universitaire de Nice, FHU OncoAge, Universit\u00e9 C\u00f4te d\u2019Azur, 06000 Nice, France"},{"name":"Team 4, Institute of Research on Cancer and Aging, CNRS INSERM, Centre Antoine Lacassagne, Universit\u00e9 C\u00f4te d\u2019Azur, 06100 Nice, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1899-9612","authenticated-orcid":false,"given":"Jonathan","family":"Benzaquen","sequence":"additional","affiliation":[{"name":"Team 4, Institute of Research on Cancer and Aging, CNRS INSERM, Centre Antoine Lacassagne, Universit\u00e9 C\u00f4te d\u2019Azur, 06100 Nice, France"},{"name":"Department of Pulmonary Medicine and Oncology, Centre Hospitalier Universitaire de Nice, FHU OncoAge, Universit\u00e9 C\u00f4te d\u2019Azur, 06000 Nice, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0563-8711","authenticated-orcid":false,"given":"Paul","family":"Tourniaire","sequence":"additional","affiliation":[{"name":"Epione Team, Inria, Universit\u00e9 C\u00f4te d\u2019Azur, 06220 Sophia Antipolis, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5916-534X","authenticated-orcid":false,"given":"Simon","family":"Heeke","sequence":"additional","affiliation":[{"name":"Department of Thoracic\/Head and Neck Medical Oncology, University of Texas MD Anderson Cancer Center, Houston, TX 77030, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Nicholas","family":"Ayache","sequence":"additional","affiliation":[{"name":"Epione Team, Inria, Universit\u00e9 C\u00f4te d\u2019Azur, 06220 Sophia Antipolis, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Herv\u00e9","family":"Delingette","sequence":"additional","affiliation":[{"name":"Epione Team, Inria, Universit\u00e9 C\u00f4te d\u2019Azur, 06220 Sophia Antipolis, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Elodie","family":"Long-Mira","sequence":"additional","affiliation":[{"name":"Laboratory of Clinical and Experimental Pathology, Centre Hospitalier Universitaire de Nice, FHU OncoAge, Universit\u00e9 C\u00f4te d\u2019Azur, 06000 Nice, France"},{"name":"Hospital-Related Biobank (BB-0033-00025), Centre Hospitalier Universitaire de Nice, FHU OncoAge, Universit\u00e9 C\u00f4te d\u2019Azur, 06000 Nice, France"},{"name":"Team 4, Institute of Research on Cancer and Aging, CNRS INSERM, Centre Antoine Lacassagne, Universit\u00e9 C\u00f4te d\u2019Azur, 06100 Nice, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Sandra","family":"Lassalle","sequence":"additional","affiliation":[{"name":"Laboratory of Clinical and Experimental Pathology, Centre Hospitalier Universitaire de Nice, FHU OncoAge, Universit\u00e9 C\u00f4te d\u2019Azur, 06000 Nice, France"},{"name":"Hospital-Related Biobank (BB-0033-00025), Centre Hospitalier Universitaire de Nice, FHU OncoAge, Universit\u00e9 C\u00f4te d\u2019Azur, 06000 Nice, France"},{"name":"Team 4, Institute of Research on Cancer and Aging, CNRS INSERM, Centre Antoine Lacassagne, Universit\u00e9 C\u00f4te d\u2019Azur, 06100 Nice, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Marame","family":"Hamila","sequence":"additional","affiliation":[{"name":"Laboratory of Clinical and Experimental Pathology, Centre Hospitalier Universitaire de Nice, FHU OncoAge, Universit\u00e9 C\u00f4te d\u2019Azur, 06000 Nice, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Julien","family":"Fayada","sequence":"additional","affiliation":[{"name":"Hospital-Related Biobank (BB-0033-00025), Centre Hospitalier Universitaire de Nice, FHU OncoAge, Universit\u00e9 C\u00f4te d\u2019Azur, 06000 Nice, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Josiane","family":"Otto","sequence":"additional","affiliation":[{"name":"Department of Oncology, Antoine Lacassagne Center, Universit\u00e9 C\u00f4te d\u2019Azur, 06100 Nice, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Charlotte","family":"Cohen","sequence":"additional","affiliation":[{"name":"Department of Thoracic Surgery, Centre Hospitalier Universitaire de Nice, FHU OncoAge, Universit\u00e9 C\u00f4te d\u2019Azur, 06000 Nice, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5813-0551","authenticated-orcid":false,"given":"Abel","family":"Gomez-Caro","sequence":"additional","affiliation":[{"name":"Department of Thoracic Surgery, Centre Hospitalier Universitaire de Nice, FHU OncoAge, Universit\u00e9 C\u00f4te d\u2019Azur, 06000 Nice, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Jean-Philippe","family":"Berthet","sequence":"additional","affiliation":[{"name":"Team 4, Institute of Research on Cancer and Aging, CNRS INSERM, Centre Antoine Lacassagne, Universit\u00e9 C\u00f4te d\u2019Azur, 06100 Nice, France"},{"name":"Department of Thoracic Surgery, Centre Hospitalier Universitaire de Nice, FHU OncoAge, Universit\u00e9 C\u00f4te d\u2019Azur, 06000 Nice, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-0846-9941","authenticated-orcid":false,"given":"Charles-Hugo","family":"Marquette","sequence":"additional","affiliation":[{"name":"Team 4, Institute of Research on Cancer and Aging, CNRS INSERM, Centre Antoine Lacassagne, Universit\u00e9 C\u00f4te d\u2019Azur, 06100 Nice, France"},{"name":"Department of Pulmonary Medicine and Oncology, Centre Hospitalier Universitaire de Nice, FHU OncoAge, Universit\u00e9 C\u00f4te d\u2019Azur, 06000 Nice, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"V\u00e9ronique","family":"Hofman","sequence":"additional","affiliation":[{"name":"Laboratory of Clinical and Experimental Pathology, Centre Hospitalier Universitaire de Nice, FHU OncoAge, Universit\u00e9 C\u00f4te d\u2019Azur, 06000 Nice, France"},{"name":"Hospital-Related Biobank (BB-0033-00025), Centre Hospitalier Universitaire de Nice, FHU OncoAge, Universit\u00e9 C\u00f4te d\u2019Azur, 06000 Nice, France"},{"name":"Team 4, Institute of Research on Cancer and Aging, CNRS INSERM, Centre Antoine Lacassagne, Universit\u00e9 C\u00f4te d\u2019Azur, 06100 Nice, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5446-6197","authenticated-orcid":false,"given":"Christophe","family":"Bontoux","sequence":"additional","affiliation":[{"name":"Laboratory of Clinical and Experimental Pathology, Centre Hospitalier Universitaire de Nice, FHU OncoAge, Universit\u00e9 C\u00f4te d\u2019Azur, 06000 Nice, France"},{"name":"Hospital-Related Biobank (BB-0033-00025), Centre Hospitalier Universitaire de Nice, FHU OncoAge, Universit\u00e9 C\u00f4te d\u2019Azur, 06000 Nice, France"},{"name":"Team 4, Institute of Research on Cancer and Aging, CNRS INSERM, Centre Antoine Lacassagne, Universit\u00e9 C\u00f4te d\u2019Azur, 06100 Nice, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0431-9353","authenticated-orcid":false,"given":"Paul","family":"Hofman","sequence":"additional","affiliation":[{"name":"Laboratory of Clinical and Experimental Pathology, Centre Hospitalier Universitaire de Nice, FHU OncoAge, Universit\u00e9 C\u00f4te d\u2019Azur, 06000 Nice, France"},{"name":"Hospital-Related Biobank (BB-0033-00025), Centre Hospitalier Universitaire de Nice, FHU OncoAge, Universit\u00e9 C\u00f4te d\u2019Azur, 06000 Nice, France"},{"name":"Team 4, Institute of Research on Cancer and Aging, CNRS INSERM, Centre Antoine Lacassagne, Universit\u00e9 C\u00f4te d\u2019Azur, 06100 Nice, France"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2022,3,29]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Miura, K., Shimizu, K., Ide, S., Mishima, S., Matsuoka, S., Takeda, T., Eguchi, T., Hamanaka, K., and Uehara, T. (2021). A Novel Strategy for the Diagnosis of Pulmonary High-Grade Neuroendocrine Tumor. Diagnostics, 11.","DOI":"10.3390\/diagnostics11111945"},{"key":"ref_2","unstructured":"WHO Classification of Tumours Editorial Board (2021). Thoracic Tumours, International Agency for Research on Cancer."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1770","DOI":"10.1038\/s41379-018-0110-y","article-title":"A common classification framework for neuroendocrine neoplasms: An International Agency for Research on Cancer (IARC) and World Health Organization (WHO) expert consensus proposal","volume":"31","author":"Rindi","year":"2018","journal-title":"Mod. Pathol."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1628","DOI":"10.5858\/2009-0583-RAR.1","article-title":"Neuroendocrine tumors of the lung: An update","volume":"134","author":"Rekhtman","year":"2010","journal-title":"Arch. Pathol. Lab Med."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"469","DOI":"10.1016\/j.thorsurg.2021.05.005","article-title":"Neuroendocrine Tumors of the Lung","volume":"31","author":"Randhawa","year":"2021","journal-title":"Thorac. Surg. Clin."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"3407","DOI":"10.1038\/s41467-019-11276-9","article-title":"Integrative and comparative genomic analyses identify clinically relevant pulmonary carcinoid groups and unveil the supra-carcinoids","volume":"10","author":"Alcala","year":"2019","journal-title":"Nat. Commun."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Chen, H., Ishihara, M., Horita, N., Kazahari, H., Ochiai, R., Tanzawa, S., Honda, T., Ichikawa, Y., Watanabe, K., and Seki, N. (2021). Effect of Adjuvant and Palliative Chemotherapy in Large Cell Neuroendocrine Carcinoma of the Lung: A Systematic Review and Meta-Analysis. Cancers, 13.","DOI":"10.3390\/cancers13235948"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Plaja, A., Moran, T., Carcereny, E., Saigi, M., Hernandez, A., Cucurull, M., and Dom\u00e8nech, M. (2021). Small-Cell Lung Cancer Long-Term Survivor Patients: How to Find a Needle in a Haystack?. Int. J. Mol. Sci., 22.","DOI":"10.3390\/ijms222413508"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"36","DOI":"10.1038\/s41379-021-00943-2","article-title":"Lung neuroendocrine neoplasms: Recent progress and persistent challenges","volume":"35","author":"Rekhtman","year":"2022","journal-title":"Mod. Pathol."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"860","DOI":"10.21037\/tlcr.2020.02.13","article-title":"Multiple faces of pulmonary large cell neuroendocrine carcinoma: Update with a focus on practical approach to diagnosis","volume":"9","author":"Baine","year":"2020","journal-title":"Transl. Lung Cancer Res."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Sonkin, D., Thomas, A., and Teicher, B.A. (2019). Are neuroendocrine negative small cell lung cancer and large cell neuroendocrine carcinoma with WT RB1 two faces of the same entity?. Lung Cancer Manag., 8.","DOI":"10.2217\/lmt-2019-0005"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"334","DOI":"10.1016\/j.jtho.2016.12.004","article-title":"The Use of Immunohistochemistry Improves the Diagnosis of Small Cell Lung Cancer and Its Differential Diagnosis. An International Reproducibility Study in a Demanding Set of Cases","volume":"12","author":"Thunnissen","year":"2017","journal-title":"J. Thorac. Oncol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"356","DOI":"10.1111\/j.1365-2559.2010.03486.x","article-title":"Small cell carcinoma of the lung and large cell neuroendocrine carcinoma interobserver variability","volume":"56","author":"Willemsen","year":"2010","journal-title":"Histopathology"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"425","DOI":"10.1016\/j.jtho.2016.11.2222","article-title":"Neuroendocrine Tumors of the Lung: Current Challenges and Advances in the Diagnosis and Management of Well-Differentiated Disease","volume":"12","author":"Hendifar","year":"2017","journal-title":"J. Thorac. Oncol."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"31","DOI":"10.1038\/s41591-021-01614-0","article-title":"AI in health and medicine","volume":"28","author":"Rajpurkar","year":"2022","journal-title":"Nat. Med."},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Niazi, M.K.K., Parwani, A.V., and Gurcan, M.N. (2019). Digital pathology and artificial intelligence. Lancet Oncol., 20.","DOI":"10.1016\/S1470-2045(19)30154-8"},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"775","DOI":"10.1038\/s41591-021-01343-4","article-title":"Deep learning in histopathology: The path to the clinic","volume":"27","author":"Litjens","year":"2021","journal-title":"Nat. Med."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"2199","DOI":"10.1001\/jama.2017.14585","article-title":"Diagnostic Assessment of Deep Learning Algorithms for Detection of Lymph Node Metastases in Women with Breast Cancer","volume":"318","author":"Veta","year":"2017","journal-title":"JAMA"},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"550","DOI":"10.1109\/TMI.2018.2867350","article-title":"From Detection of Individual Metastases to Classification of Lymph Node Status at the Patient Level: The CAMELYON17 Challenge","volume":"38","author":"Bandi","year":"2019","journal-title":"IEEE Trans. Med. Imaging"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"1559","DOI":"10.1038\/s41591-018-0177-5","article-title":"Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning","volume":"24","author":"Coudray","year":"2018","journal-title":"Nat. Med."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"757","DOI":"10.1093\/jamia\/ocz230","article-title":"Classifying non-small cell lung cancer types and transcriptomic subtypes using convolutional neural networks","volume":"27","author":"Yu","year":"2020","journal-title":"J. Am. Med. Inform. Assoc."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Sadhwani, A., Chang, H.W., Behrooz, A., Brown, T., Auvigne-Flament, I., Patel, H., Findlater, R., Velez, V., Tan, F., and Tekiela, K. (2021). Comparative analysis of machine learning approaches to classify tumor mutation burden in lung adenocarcinoma using histopathology images. Sci. Rep., 11.","DOI":"10.1038\/s41598-021-95747-4"},{"key":"ref_23","doi-asserted-by":"crossref","unstructured":"Carrillo-Perez, F., Morales, J.C., Castillo-Secilla, D., Molina-Castro, Y., Guillen, A., Rojas, I., and Herrera, L.J. (2021). Non-small-cell lung cancer classification via RNA-Seq and histology imaging probability fusion. BMC Bioinform., 22.","DOI":"10.1186\/s12859-021-04376-1"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"117","DOI":"10.5858\/arpa.2020-0520-OA","article-title":"A Deep Learning Convolutional Neural Network Can Differentiate Between Helicobacter Pylori Gastritis and Autoimmune Gastritis with Results Comparable to Gastrointestinal Pathologists","volume":"146","author":"Franklin","year":"2022","journal-title":"Arch. Pathol. Lab Med."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Guo, J., Hou, L., Zhang, W., Dong, Z., Zhang, L., and Wu, C. (2021). Improving differential diagnosis of pulmonary large cell neuroendocrine carcinoma and small cell lung cancer via a transcriptomic, biological pathway-based machine learning model. Transl. Oncol., 14.","DOI":"10.1016\/j.tranon.2021.101222"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"32","DOI":"10.4103\/jpi.jpi_37_20","article-title":"Comparing Deep Learning and Immunohistochemistry in Determining the Site of Origin for Well-Differentiated Neuroendocrine Tumors","volume":"11","author":"Redemann","year":"2020","journal-title":"J. Pathol. Inform."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"426","DOI":"10.1111\/cyt.12829","article-title":"Feasibility of a deep learning algorithm to distinguish large cell neuroendocrine from small cell lung carcinoma in cytology specimens","volume":"31","author":"Gonzalez","year":"2020","journal-title":"Cytopathology"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Bulloni, M., Sandrini, G., Stacchiotti, I., Barberis, M., Calabrese, F., Carvalho, L., Fontanini, G., Al\u00ec, G., Fortarezza, F., and Hofman, P. (2021). Automated Analysis of Proliferating Cells Spatial Organisation Predicts Prognosis in Lung Neuroendocrine Neoplasms. Cancers, 13.","DOI":"10.3390\/cancers13194875"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Govind, D., Jen, K.Y., Matsukuma, K., Gao, G., Olson, K.A., Gui, D., Wilding, G., Border, S.P., and Sarder, P. (2020). Improving the accuracy of gastrointestinal neuroendocrine tumor grading with deep learning. Sci. Rep., 10.","DOI":"10.1038\/s41598-020-67880-z"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1007\/s00428-020-03015-z","article-title":"Morphologic and molecular classification of lung neuroendocrine neoplasms","volume":"478","author":"Metovic","year":"2021","journal-title":"Virchows Arch."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"90","DOI":"10.1053\/j.semdp.2021.03.002","article-title":"Recent advances and current controversies in lung neuroendocrine neoplasms","volume":"38","author":"Metovic","year":"2021","journal-title":"Semin. Diagn. Pathol."}],"container-title":["Cancers"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-6694\/14\/7\/1740\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T22:45:49Z","timestamp":1760136349000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-6694\/14\/7\/1740"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,3,29]]},"references-count":31,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2022,4]]}},"alternative-id":["cancers14071740"],"URL":"https:\/\/doi.org\/10.3390\/cancers14071740","relation":{},"ISSN":["2072-6694"],"issn-type":[{"value":"2072-6694","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,3,29]]}}}