{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,29]],"date-time":"2025-10-29T13:54:55Z","timestamp":1761746095142,"version":"3.41.2"},"reference-count":52,"publisher":"American Society of Clinical Oncology (ASCO)","issue":"8","content-domain":{"domain":["ascopubs.org"],"crossmark-restriction":true},"short-container-title":["JCO Clin Cancer Inform"],"published-print":{"date-parts":[[2024,12]]},"abstract":"<jats:sec>\n            <jats:title>PURPOSE<\/jats:title>\n            <jats:p>Thyroid nodules are common in the general population, and assessing their malignancy risk is the initial step in care. Surgical exploration remains the sole definitive option for indeterminate nodules. Extensive database access is crucial for improving this initial assessment. Our objective was to develop an automated process using convolutional neural networks (CNNs) to extract and structure biomedical insights from electronic health reports (EHRs) in a large thyroid pathology cohort.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>MATERIALS AND METHODS<\/jats:title>\n            <jats:p>We randomly selected 1,500 patients with thyroid pathology from our cohort for model development and an additional 100 for testing. We then divided the cohort of 1,500 patients into training (70%) and validation (30%) sets. We used EHRs from initial surgeon visits, preanesthesia visits, ultrasound, surgery, and anatomopathology reports. We selected 42 variables of interest and had them manually annotated by a clinical expert. We developed RUBY-THYRO using six distinct CNN models from SpaCy, supplemented with keyword extraction rules and postprocessing. Evaluation against a gold standard database included calculating precision, recall, and F1 score.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>RESULTS<\/jats:title>\n            <jats:p>Performance remained consistent across the test and validation sets, with the majority of variables (30\/42) achieving performance metrics exceeding 90% for all metrics in both sets. Results differed according to the variables; pathologic tumor stage score achieved 100% in precision, recall, and F1 score, versus 45%, 28%, and 32% for the number of nodules in the test set, respectively. Surgical and preanesthesia reports demonstrated particularly high performance.<\/jats:p>\n          <\/jats:sec>\n          <jats:sec>\n            <jats:title>CONCLUSION<\/jats:title>\n            <jats:p>Our study successfully implemented a CNN-based natural language processing (NLP) approach for extracting and structuring data from various EHRs in thyroid pathology. This highlights the potential of artificial intelligence-driven NLP techniques for extensive and cost-effective data extraction, paving the way for creating comprehensive, hospital-wide data warehouses.<\/jats:p>\n          <\/jats:sec>","DOI":"10.1200\/cci.23.00263","type":"journal-article","created":{"date-parts":[[2024,12,10]],"date-time":"2024-12-10T21:00:23Z","timestamp":1733864423000},"update-policy":"https:\/\/doi.org\/10.1200\/crossmark","source":"Crossref","is-referenced-by-count":1,"title":["Enhancing Thyroid Pathology With Artificial Intelligence: Automated Data Extraction From Electronic Health Reports Using RUBY"],"prefix":"10.1200","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-7964-8488","authenticated-orcid":true,"given":"Dorian","family":"Culi\u00e9","sequence":"first","affiliation":[{"name":"Cervico-Facial Oncology Surgical Department, University Institute of Face and Neck, Centre Antoine Lacassagne University of C\u00f4te d'Azur, Nice, France"},{"name":"Department of Epidemiology, Biostatistics and Health Data, Centre Antoine Lacassagne, University of C\u00f4te d'Azur, Nice, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-5104-9507","authenticated-orcid":true,"given":"Renaud","family":"Schiappa","sequence":"additional","affiliation":[{"name":"Department of Epidemiology, Biostatistics and Health Data, Centre Antoine Lacassagne, University of C\u00f4te d'Azur, Nice, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9235-9675","authenticated-orcid":true,"given":"Sara","family":"Contu","sequence":"additional","affiliation":[{"name":"Department of Epidemiology, Biostatistics and Health Data, Centre Antoine Lacassagne, University of C\u00f4te d'Azur, Nice, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Eva","family":"Seutin","sequence":"additional","affiliation":[{"name":"Department of Epidemiology, Biostatistics and Health Data, Centre Antoine Lacassagne, University of C\u00f4te d'Azur, Nice, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-8723-2507","authenticated-orcid":true,"given":"Tanguy","family":"Pace-Loscos","sequence":"additional","affiliation":[{"name":"Department of Epidemiology, Biostatistics and Health Data, Centre Antoine Lacassagne, University of C\u00f4te d'Azur, Nice, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Gilles","family":"Poissonnet","sequence":"additional","affiliation":[{"name":"Cervico-Facial Oncology Surgical Department, University Institute of Face and Neck, Centre Antoine Lacassagne University of C\u00f4te d'Azur, Nice, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Agathe","family":"Villarme","sequence":"additional","affiliation":[{"name":"Cervico-Facial Oncology Surgical Department, University Institute of Face and Neck, Centre Antoine Lacassagne University of C\u00f4te d'Azur, Nice, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Alexandre","family":"Bozec","sequence":"additional","affiliation":[{"name":"Cervico-Facial Oncology Surgical Department, University Institute of Face and Neck, Centre Antoine Lacassagne University of C\u00f4te d'Azur, Nice, France"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Emmanuel","family":"Chamorey","sequence":"additional","affiliation":[{"name":"Department of Epidemiology, Biostatistics and Health Data, Centre Antoine Lacassagne, University of C\u00f4te d'Azur, Nice, France"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"233","reference":[{"key":"e_1_3_5_2_2","doi-asserted-by":"publisher","DOI":"10.21037\/gs.2018.12.06"},{"key":"e_1_3_5_3_2","doi-asserted-by":"publisher","DOI":"10.1016\/S2213-8587(20)30115-7"},{"key":"e_1_3_5_4_2","doi-asserted-by":"publisher","DOI":"10.1177\/1073274817729231"},{"key":"e_1_3_5_5_2","doi-asserted-by":"publisher","DOI":"10.1159\/000478927"},{"key":"e_1_3_5_6_2","doi-asserted-by":"publisher","DOI":"10.1159\/000451071"},{"key":"e_1_3_5_7_2","doi-asserted-by":"publisher","DOI":"10.1089\/thy.2015.0020"},{"key":"e_1_3_5_8_2","doi-asserted-by":"publisher","DOI":"10.1159\/000339959"},{"key":"e_1_3_5_9_2","doi-asserted-by":"publisher","DOI":"10.1093\/ejendo\/lvad017"},{"key":"e_1_3_5_10_2","doi-asserted-by":"publisher","DOI":"10.3390\/curroncol30050361"},{"key":"e_1_3_5_11_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2008.12.005"},{"key":"e_1_3_5_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jpainsymman.2018.02.016"},{"key":"e_1_3_5_13_2","doi-asserted-by":"publisher","DOI":"10.4103\/2153-3539.159215"},{"key":"e_1_3_5_14_2","doi-asserted-by":"crossref","unstructured":"Gao S Young MT Qiu JX et al: Supporting information for hierarchical attention networks for information extraction from cancer pathology reports preprocessing procedures 2023. https:\/\/academic.oup.com\/jamia\/article\/25\/3\/321\/4636780","DOI":"10.1093\/jamia\/ocx131"},{"key":"e_1_3_5_15_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.artmed.2019.101726"},{"key":"e_1_3_5_16_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patter.2023.100726"},{"key":"e_1_3_5_17_2","article-title":"RUBY: Natural language processing of French electronic medical records for breast cancer research","author":"Schiappa R","unstructured":"Schiappa R, Contu S, Culie D, et al: RUBY: Natural language processing of French electronic medical records for breast cancer research. JCO Clin Cancer Inform 10.1200\/CCI.21.00199","journal-title":"JCO Clin Cancer Inform"},{"key":"e_1_3_5_18_2","doi-asserted-by":"publisher","DOI":"10.1016\/S2213-8587(22)00139-5"},{"key":"e_1_3_5_19_2","doi-asserted-by":"publisher","DOI":"10.3390\/ijerph191912200"},{"key":"e_1_3_5_20_2","article-title":"Validation of RUBY for breast cancer knowledge extraction from a large French electronic medical record system","author":"Schiappa R","unstructured":"Schiappa R, Contu S, Culie D, et al: Validation of RUBY for breast cancer knowledge extraction from a large French electronic medical record system. JCO Clin Cancer Inform 10.1200\/CCI.22.00130","journal-title":"JCO Clin Cancer Inform"},{"key":"e_1_3_5_21_2","unstructured":"Honnibal M Montani I: spaCy 2: Natural language understanding with Bloom embeddings convolutional neural networks and incremental parsing. To Appear 7:411-420 2017"},{"key":"e_1_3_5_22_2","unstructured":"Stenetorp P Pyysalo S Topi\u0107 G et al: BRAT: A web-based tool for NLP-assisted text annotation. Proceedings of the Demonstrations at the 13th Conference of the European Chapter of the Association for Computational Linguistics Avignon France April 2012 pp 102-1077"},{"key":"e_1_3_5_23_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10489-021-02635-5"},{"key":"e_1_3_5_24_2","doi-asserted-by":"publisher","DOI":"10.3389\/fmed.2022.906554"},{"key":"e_1_3_5_25_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijporl.2023.111698"},{"key":"e_1_3_5_26_2","article-title":"Natural language processing for automated quantification of brain metastases reported in free-text radiology reports","author":"Senders JT","unstructured":"Senders JT, Karhade AV, Cote DJ, et al: Natural language processing for automated quantification of brain metastases reported in free-text radiology reports. JCO Clin Cancer Inform 10.1200\/CCI.18.00138","journal-title":"JCO Clin Cancer Inform"},{"key":"e_1_3_5_27_2","doi-asserted-by":"publisher","DOI":"10.1038\/s43856-022-00157-w"},{"key":"e_1_3_5_28_2","first-page":"55","article-title":"Extraction of temporal structures for clinical events in unlabeled free-text electronic health records in Russian","volume":"287","author":"Funkner AA","year":"2021","unstructured":"Funkner AA, Zhurman DA, Kovalchuk SV: Extraction of temporal structures for clinical events in unlabeled free-text electronic health records in Russian. Stud Health Technol Inform 287:55-56, 2021","journal-title":"Stud Health Technol Inform"},{"key":"e_1_3_5_29_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijmedinf.2021.104615"},{"key":"e_1_3_5_30_2","first-page":"393","article-title":"Identifying diabetes in clinical notes in Hebrew: A novel text classification approach based on word embedding","volume":"264","author":"Topaz M","year":"2019","unstructured":"Topaz M, Murga L, Grossman C, et al: Identifying diabetes in clinical notes in Hebrew: A novel text classification approach based on word embedding. Stud Health Technol Inform 264:393-397, 2019","journal-title":"Stud Health Technol Inform"},{"key":"e_1_3_5_31_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.bjps.2023.05.035"},{"key":"e_1_3_5_32_2","first-page":"795","article-title":"Bridging the granularity gap in family history information extracted from clinical narratives","volume":"2022","author":"Moon S","year":"2022","unstructured":"Moon S, Wang L, Chen X, et al: Bridging the granularity gap in family history information extracted from clinical narratives. AMIA Annu Symp Proc 2022:795-804, 2022","journal-title":"AMIA Annu Symp Proc"},{"key":"e_1_3_5_33_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.isci.2023.108163"},{"key":"e_1_3_5_34_2","doi-asserted-by":"publisher","DOI":"10.3389\/fdgth.2022.793316"},{"key":"e_1_3_5_35_2","article-title":"Deep learning for natural language processing in urology: State-of-the-art automated extraction of detailed pathologic prostate cancer data from narratively written electronic health records","author":"Leyh-Bannurah S-R","unstructured":"Leyh-Bannurah S-R, Tian Z, Karakiewicz PI, et al: Deep learning for natural language processing in urology: State-of-the-art automated extraction of detailed pathologic prostate cancer data from narratively written electronic health records. JCO Clin Cancer Inform 10.1200\/CCI.18.00080","journal-title":"JCO Clin Cancer Inform"},{"key":"e_1_3_5_36_2","doi-asserted-by":"publisher","DOI":"10.15265\/IYS-2016-s006"},{"key":"e_1_3_5_37_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jocn.2022.01.015"},{"key":"e_1_3_5_38_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.arth.2020.09.029"},{"key":"e_1_3_5_39_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.arth.2022.10.031"},{"key":"e_1_3_5_40_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11605-022-05282-4"},{"key":"e_1_3_5_41_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0284965"},{"key":"e_1_3_5_42_2","doi-asserted-by":"publisher","DOI":"10.1177\/00031348221117039"},{"key":"e_1_3_5_43_2","doi-asserted-by":"publisher","DOI":"10.2106\/JBJS.22.00567"},{"key":"e_1_3_5_44_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-019-0208-8"},{"key":"e_1_3_5_45_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10916-019-1279-4"},{"key":"e_1_3_5_46_2","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocab243"},{"key":"e_1_3_5_47_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-019-0781-4"},{"key":"e_1_3_5_48_2","article-title":"Do neural information extraction algorithms generalize across institutions?","author":"Santus E","unstructured":"Santus E, Li C, Yala A, et al: Do neural information extraction algorithms generalize across institutions? JCO Clin Cancer Inform 10.1200\/CCI.18.00160","journal-title":"JCO Clin Cancer Inform"},{"key":"e_1_3_5_49_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.techfore.2015.12.019"},{"key":"e_1_3_5_50_2","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pone.0256152"},{"key":"e_1_3_5_51_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12874-022-01665-y"},{"key":"e_1_3_5_52_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.compbiomed.2023.106649"},{"key":"e_1_3_5_53_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-017-0464-y"}],"container-title":["JCO Clinical Cancer Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/ascopubs.org\/doi\/pdfdirect\/10.1200\/CCI.23.00263","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,12,10]],"date-time":"2024-12-10T21:00:28Z","timestamp":1733864428000},"score":1,"resource":{"primary":{"URL":"https:\/\/ascopubs.org\/doi\/10.1200\/CCI.23.00263"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,12]]},"references-count":52,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2024,12]]}},"alternative-id":["10.1200\/CCI.23.00263"],"URL":"https:\/\/doi.org\/10.1200\/cci.23.00263","relation":{},"ISSN":["2473-4276"],"issn-type":[{"type":"electronic","value":"2473-4276"}],"subject":[],"published":{"date-parts":[[2024,12]]},"assertion":[{"value":"2023-12-14","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-07-11","order":1,"name":"revised","label":"Revised","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-09-25","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-12-10","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e2300263"}}