{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,11,26]],"date-time":"2025-11-26T05:07:18Z","timestamp":1764133638407,"version":"3.41.2"},"reference-count":34,"publisher":"Frontiers Media SA","license":[{"start":{"date-parts":[[2023,6,14]],"date-time":"2023-06-14T00:00:00Z","timestamp":1686700800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["frontiersin.org"],"crossmark-restriction":true},"short-container-title":["Front. Digit. Health"],"abstract":"<jats:sec><jats:title>Objectives<\/jats:title><jats:p>The objective of this study is the exploration of Artificial Intelligence and Natural Language Processing techniques to support the automatic assignment of the four Response Evaluation Criteria in Solid Tumors (RECIST) scales based on radiology reports. We also aim at evaluating how languages and institutional specificities of Swiss teaching hospitals are likely to affect the quality of the classification in French and German languages.<\/jats:p><\/jats:sec><jats:sec><jats:title>Methods<\/jats:title><jats:p>In our approach, 7 machine learning methods were evaluated to establish a strong baseline. Then, robust models were built, fine-tuned according to the language (French and German), and compared with the expert annotation.<\/jats:p><\/jats:sec><jats:sec><jats:title>Results<\/jats:title><jats:p>The best strategies yield average F1-scores of 90% and 86% respectively for the 2-classes (Progressive\/Non-progressive) and the 4-classes (Progressive Disease, Stable Disease, Partial Response, Complete Response) RECIST classification tasks.<\/jats:p><\/jats:sec><jats:sec><jats:title>Conclusions<\/jats:title><jats:p>These results are competitive with the manual labeling as measured by Matthew's correlation coefficient and Cohen's Kappa (79% and 76%). On this basis, we confirm the capacity of specific models to generalize on new unseen data and we assess the impact of using Pre-trained Language Models (PLMs) on the accuracy of the classifiers.<\/jats:p><\/jats:sec>","DOI":"10.3389\/fdgth.2023.1195017","type":"journal-article","created":{"date-parts":[[2023,6,14]],"date-time":"2023-06-14T05:41:57Z","timestamp":1686721317000},"update-policy":"https:\/\/doi.org\/10.3389\/crossmark-policy","source":"Crossref","is-referenced-by-count":3,"title":["Multilingual RECIST classification of radiology reports using supervised learning"],"prefix":"10.3389","volume":"5","author":[{"given":"Luc","family":"Mottin","sequence":"first","affiliation":[]},{"given":"Jean-Philippe","family":"Goldman","sequence":"additional","affiliation":[]},{"given":"Christoph","family":"J\u00e4ggli","sequence":"additional","affiliation":[]},{"given":"Rita","family":"Achermann","sequence":"additional","affiliation":[]},{"given":"Julien","family":"Gobeill","sequence":"additional","affiliation":[]},{"given":"Julien","family":"Knafou","sequence":"additional","affiliation":[]},{"given":"Julien","family":"Ehrsam","sequence":"additional","affiliation":[]},{"given":"Alexandre","family":"Wicky","sequence":"additional","affiliation":[]},{"given":"Camille L.","family":"G\u00e9rard","sequence":"additional","affiliation":[]},{"given":"Tanja","family":"Schwenk","sequence":"additional","affiliation":[]},{"given":"M\u00e9linda","family":"Charrier","sequence":"additional","affiliation":[]},{"given":"Petros","family":"Tsantoulis","sequence":"additional","affiliation":[]},{"given":"Christian","family":"Lovis","sequence":"additional","affiliation":[]},{"given":"Alexander","family":"Leichtle","sequence":"additional","affiliation":[]},{"given":"Michael K.","family":"Kiessling","sequence":"additional","affiliation":[]},{"given":"Olivier","family":"Michielin","sequence":"additional","affiliation":[]},{"given":"Sylvain","family":"Pradervand","sequence":"additional","affiliation":[]},{"given":"Vasiliki","family":"Foufi","sequence":"additional","affiliation":[]},{"given":"Patrick","family":"Ruch","sequence":"additional","affiliation":[]}],"member":"1965","published-online":{"date-parts":[[2023,6,14]]},"reference":[{"key":"B1","doi-asserted-by":"publisher","first-page":"228","DOI":"10.1016\/j.ejca.2008.10.026","article-title":"New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1)","volume":"45","author":"Eisenhauer","year":"2009","journal-title":"Eur J Cancer"},{"key":"B2","doi-asserted-by":"publisher","first-page":"132","DOI":"10.1016\/j.ejca.2016.03.081","article-title":"RECIST 1.1-Update And clarification: from the RECIST committee","volume":"62","author":"Schwartz","year":"2016","journal-title":"Eur J Cancer"},{"key":"B3","doi-asserted-by":"publisher","first-page":"488","DOI":"10.1197\/jamia.m2082","article-title":"Advancing biomedical image retrieval: development and analysis of a test collection","volume":"13","author":"Hersh","year":"2006","journal-title":"J Am Med Inform Assoc"},{"first-page":"1","year":"2016","author":"Chodey","key":"B4"},{"key":"B5","doi-asserted-by":"publisher","first-page":"72","DOI":"10.1016\/j.artmed.2018.05.006","article-title":"Automatic classification of radiological reports for clinical care","volume":"91","author":"Gerevini","year":"2018","journal-title":"Artif Intell Med"},{"key":"B6","doi-asserted-by":"publisher","first-page":"e17984","DOI":"10.2196\/17984","article-title":"Clinical text data in machine learning: systematic review","volume":"8","author":"Spasic","year":"2020","journal-title":"JMIR Med Inform"},{"key":"B7","doi-asserted-by":"publisher","first-page":"680","DOI":"10.1200\/CCI.20.00020","article-title":"Natural language processing to ascertain cancer outcomes from medical oncologist notes","volume":"4","author":"Kehl","year":"2020","journal-title":"JCO Clin Cancer Inform"},{"key":"B8","doi-asserted-by":"publisher","first-page":"379","DOI":"10.1200\/CCI.20.00173","article-title":"Natural language processing to identify cancer treatments with electronic medical records","volume":"5","author":"Zeng","year":"2021","journal-title":"JCO Clin Cancer Inform"},{"key":"B9","doi-asserted-by":"publisher","first-page":"885","DOI":"10.1200\/CCI.20.00072","article-title":"Machine learning in oncology: methods, applications, and challenges","volume":"4","author":"Bertsimas","year":"2020","journal-title":"JCO Clin Cancer Inform"},{"volume-title":"Summary of the HIPAA privacy rule: HIPAA compliance assistance","year":"2003","key":"B10"},{"key":"B11","doi-asserted-by":"publisher","first-page":"419","DOI":"10.3233\/IDA-2006-10503","article-title":"Learning from ambiguously labeled examples","volume":"10","author":"H\u00fcllermeier","year":"2006","journal-title":"International Symposium on Intelligent Data Analysis"},{"key":"B12","doi-asserted-by":"publisher","first-page":"318","DOI":"10.1148\/radiol.2018171820","article-title":"Current applications and future impact of machine learning in radiology","volume":"288","author":"Choy","year":"2018","journal-title":"Radiology"},{"key":"B13","doi-asserted-by":"publisher","first-page":"3449","DOI":"10.1002\/mp.12967","article-title":"Machine learning algorithms for outcome prediction in (chemo)radiotherapy: an empirical comparison of classifiers","volume":"45","author":"Deist","year":"2018","journal-title":"Med Phys"},{"key":"B14","doi-asserted-by":"publisher","first-page":"e1","DOI":"10.1093\/jamia\/ocv155","article-title":"Classification of radiology reports for falls in an HIV study cohort","volume":"23","author":"Bates","year":"2016","journal-title":"J Am Med Inform Assoc"},{"year":"2022","author":"Goldman","key":"B15"},{"key":"B16","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/j.jbi.2017.07.010","article-title":"Reproducibility of studies on text mining for citation screening in systematic reviews: evaluation and checklist","volume":"73","author":"Olorisade","year":"2017","journal-title":"J Biomed Inform"},{"key":"B17","doi-asserted-by":"publisher","first-page":"2825","DOI":"10.5555\/1953048.2078195","article-title":"Scikit-learn: machine learning in python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J Mach Learn Res"},{"first-page":"1237","year":"2020","author":"Zhang","key":"B18"},{"key":"B19","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1200\/CCI.19.00012","article-title":"Significant and distinctive n-grams in oncology notes: a text-mining method to analyze the effect of OpenNotes on clinical documentation","volume":"3","author":"Rahimian","year":"2019","journal-title":"JCO Clin Cancer Inform"},{"key":"B20","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1093\/gigascience\/gix020","article-title":"Using and understanding cross-validation strategies. Perspectives on Saeb et al","volume":"6","author":"Little","year":"2017","journal-title":"GigaScience"},{"key":"B21","first-page":"588","volume-title":".","author":"Rao","year":"2008"},{"key":"B22","doi-asserted-by":"publisher","first-page":"26","DOI":"10.11989\/JEST.1674-862X.80904120","article-title":"Hyperparameter optimization for machine learning models based on Bayesian optimization","volume":"17","author":"Wu","year":"2019","journal-title":"J Electron Sci Technol"},{"key":"B23","doi-asserted-by":"publisher","first-page":"189","DOI":"10.1093\/bib\/bbq073","article-title":"An empirical assessment of validation practices for molecular classifiers","volume":"12","author":"Castaldi","year":"2011","journal-title":"Brief Bioinformatics"},{"key":"B24","doi-asserted-by":"publisher","first-page":"1","DOI":"10.5281\/zenodo.3557376","article-title":"A review on evaluation metrics for data classification evaluations","volume":"5","author":"Hossin","year":"2019","journal-title":"International Journal of Data Mining & Knowledge Management Process (IJDKP)"},{"first-page":"1015","year":"2006","author":"Sokolova","key":"B25"},{"key":"B26","doi-asserted-by":"publisher","first-page":"442","DOI":"10.1016\/0005-2795(75)90109-9","article-title":"Comparison of the predicted and observed secondary structure of T4 phage lysozyme","volume":"405","author":"Matthews","year":"1975","journal-title":"Biochim Biophys Acta"},{"key":"B27","doi-asserted-by":"publisher","DOI":"10.1186\/s12864-019-6413-7","article-title":"The advantages of the matthews correlation coefficient (MCC) over F1 score and accuracy in binary classification evaluation","volume":"21","author":"Chicco","year":"2020","journal-title":"BMC Genomics"},{"key":"B28","doi-asserted-by":"publisher","first-page":"276","DOI":"10.11613\/bm.2012.031","article-title":"Interrater reliability: the kappa statistic","volume":"22","author":"McHugh","year":"2012","journal-title":"Biochem Med (Zagreb)"},{"key":"B29","doi-asserted-by":"publisher","first-page":"1247","DOI":"10.1093\/jamia\/ocz149","article-title":"Evaluating shallow and deep learning strategies for the 2018 n2c2 shared task on clinical text classification","volume":"26","author":"Oleynik","year":"2019","journal-title":"J Am Med Inform Assoc"},{"first-page":"115","year":"2016","author":"Cotik","key":"B30"},{"key":"B31","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1016\/S0933-3657(03)00052-6","article-title":"Using lexical disambiguation and named-entity recognition to improve spelling correction in the electronic patient record","volume":"29","author":"Ruch","year":"2003","journal-title":"Artif Intell Med"},{"year":"2020","author":"Le","key":"B32"},{"year":"2023","author":"Labrak","key":"B33"},{"first-page":"7203","year":"2020","author":"Martin","key":"B34"}],"container-title":["Frontiers in Digital Health"],"original-title":[],"link":[{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdgth.2023.1195017\/full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,6,14]],"date-time":"2023-06-14T05:42:01Z","timestamp":1686721321000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.frontiersin.org\/articles\/10.3389\/fdgth.2023.1195017\/full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,14]]},"references-count":34,"alternative-id":["10.3389\/fdgth.2023.1195017"],"URL":"https:\/\/doi.org\/10.3389\/fdgth.2023.1195017","relation":{},"ISSN":["2673-253X"],"issn-type":[{"type":"electronic","value":"2673-253X"}],"subject":[],"published":{"date-parts":[[2023,6,14]]},"article-number":"1195017"}}