{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,25]],"date-time":"2026-09-25T20:10:52Z","timestamp":1790367052154,"version":"4.1.0"},"reference-count":37,"publisher":"American Society of Clinical Oncology (ASCO)","issue":"1","content-domain":{"domain":["ascopubs.org"],"crossmark-restriction":true},"short-container-title":["JCO Clin Cancer Inform"],"published-print":{"date-parts":[[2026,1]]},"abstract":"<jats:sec>\n                    <jats:title>PURPOSE<\/jats:title>\n                    <jats:p>Medical free texts such as pathology reports contain valuable clinical data but are challenging to structure at scale. Traditional natural language processing approaches require extensive annotated data and training. We investigate the use of large language model (LLM) like Mistral to automatically extract three breast cancer (BC) biomarkers from pathology reports.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>MATERIALS AND METHODS<\/jats:title>\n                    <jats:p>We developed and evaluated a pipeline combining Mistral Large LLM and a postprocessing phase. The pipeline's performance was assessed both at document and patient levels. For evaluation, two data sets were used: a data set of 1,152 pathology reports associated with 150 patients with BC focused solely on biomarker values and a gold standard database containing 101 patients with metastatic BC, enriched with detailed patient and tumor characteristics and double-blind validated by clinical research assistants. We also explored the pipeline's performance according to the use of a confidence prompt (CP), a chain of thought (CoT), and few-shot examples.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>RESULTS<\/jats:title>\n                    <jats:p>Our extraction pipeline achieved F1 scores of more than 95% and both recall and precision of more than 94% for each biomarker of interest (ie, estrogen receptor, progesterone receptor and human epidermal growth factor receptor 2 status and score) at the document level. At the patient level, the F1 score decreased between 87% and 90% with a greater drop in recall (ranging between 83% and 87%) compared with precision, which remained &gt;90%. The results were similar whether the pipeline included a CP, CoT, or few-shot examples.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>CONCLUSION<\/jats:title>\n                    <jats:p>Our study provides strong evidence of the potential of LLMs like Mistral Large for extracting structured BC biomarker data from pathology reports and the potential of such methods for broader digital transformation of health care documents.<\/jats:p>\n                  <\/jats:sec>","DOI":"10.1200\/cci-25-00133","type":"journal-article","created":{"date-parts":[[2026,2,18]],"date-time":"2026-02-18T21:00:34Z","timestamp":1771448434000},"update-policy":"https:\/\/doi.org\/10.1200\/crossmark","source":"Crossref","is-referenced-by-count":4,"title":["Development and Assessment of a Pipeline for Extracting Structured Data From Free-Text Medical Reports Using a Large Language Model"],"prefix":"10.1200","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-3339-6191","authenticated-orcid":true,"given":"Enzo","family":"Joseph","sequence":"first","affiliation":[{"name":"Data Factory &amp; Analytics Department, Institut de Canc\u00e9rologie de l'Ouest, Nantes-Angers, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Paul","family":"Vallee","sequence":"additional","affiliation":[{"name":"Data Factory &amp; Analytics Department, Institut de Canc\u00e9rologie de l'Ouest, Nantes-Angers, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-1118-2271","authenticated-orcid":true,"given":"Tanguy","family":"Perennec","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Institut de Canc\u00e9rologie de l'Ouest, Nantes-Angers, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6666-0210","authenticated-orcid":true,"given":"Nicolas","family":"Wagneur","sequence":"additional","affiliation":[{"name":"Department of Radiation Oncology, Institut de Canc\u00e9rologie de l'Ouest, Nantes-Angers, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5273-0561","authenticated-orcid":true,"given":"Jean-S\u00e9bastien","family":"Frenel","sequence":"additional","affiliation":[{"name":"Oncology Department, Institut de Canc\u00e9rologie de l'Ouest, Nantes-Angers, France"},{"name":"Center for Research in Cancerology and Immunology Nantes-Angers, INSERM UMR 1232, Nantes University and Angers University, Nantes-Angers, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Mario","family":"Campone","sequence":"additional","affiliation":[{"name":"Oncology Department, Institut de Canc\u00e9rologie de l'Ouest, Nantes-Angers, France"},{"name":"Center for Research in Cancerology and Immunology Nantes-Angers, INSERM UMR 1232, Nantes University and Angers University, Nantes-Angers, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-6247-1506","authenticated-orcid":true,"given":"Fran\u00e7ois","family":"Bocquet","sequence":"additional","affiliation":[{"name":"Data Factory &amp; Analytics Department, Institut de Canc\u00e9rologie de l'Ouest, Nantes-Angers, France"},{"name":"Law and Social Change Laboratory, Faculty of Law and Political Sciences, CNRS UMR 6297, Nantes University, Nantes, France"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2361-1608","authenticated-orcid":true,"given":"Florent","family":"Le Borgne","sequence":"additional","affiliation":[{"name":"Data Factory &amp; Analytics Department, Institut de Canc\u00e9rologie de l'Ouest, Nantes-Angers, France"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"233","reference":[{"key":"e_1_3_5_2_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ejro.2024.100582"},{"key":"e_1_3_5_3_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-018-0029-1"},{"key":"e_1_3_5_4_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2019.103337"},{"key":"e_1_3_5_5_2","doi-asserted-by":"publisher","DOI":"10.1111\/1475-6773.12135"},{"key":"e_1_3_5_6_2","doi-asserted-by":"publisher","DOI":"10.1016\/S2589-7500(22)00154-6"},{"key":"e_1_3_5_7_2","doi-asserted-by":"publisher","DOI":"10.1056\/NEJMsb1609216"},{"key":"e_1_3_5_8_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2024.104720"},{"key":"e_1_3_5_9_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jclinepi.2019.07.005"},{"key":"e_1_3_5_10_2","doi-asserted-by":"publisher","DOI":"10.1136\/jamia.2010.003707"},{"key":"e_1_3_5_11_2","doi-asserted-by":"publisher","DOI":"10.2196\/18526"},{"key":"e_1_3_5_12_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2021.103712"},{"key":"e_1_3_5_13_2","first-page":"4171","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers)","author":"Devlin J","year":"2019","unstructured":"Devlin J, Chang M-W, Lee K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding. In: Burstein J, Doran C, Solorio T, eds. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers). Minneapolis, MN: Association for Computational Linguistics; 2019:4171\u20134186."},{"key":"e_1_3_5_14_2","article-title":"Development and validation of a natural language processing algorithm for extracting clinical and pathological features of breast cancer from pathology reports","author":"Munzone E","unstructured":"Munzone E, Marra A, Comotto F, et al: Development and validation of a natural language processing algorithm for extracting clinical and pathological features of breast cancer from pathology reports. JCO Clin Cancer Inform 10.1200\/CCI.24.00034","journal-title":"JCO Clin Cancer Inform"},{"key":"e_1_3_5_15_2","doi-asserted-by":"publisher","DOI":"10.1197\/jamia.M2437"},{"key":"e_1_3_5_16_2","doi-asserted-by":"publisher","DOI":"10.1093\/jamiaopen\/ooae041"},{"key":"e_1_3_5_17_2","doi-asserted-by":"publisher","DOI":"10.2196\/38125"},{"key":"e_1_3_5_18_2","doi-asserted-by":"publisher","unstructured":"Preston S Wei M Rao R et al: Towards structuring real-world data at scale: Deep learning for extracting key oncology information from clinical text with patient-level supervision. arXiv 10.48550\/arXiv.2203.10442","DOI":"10.48550\/arXiv.2203.10442"},{"key":"e_1_3_5_19_2","article-title":"Customizable natural language processing biomarker extraction tool","author":"Holmes B","unstructured":"Holmes B, Chitale D, Loving J, et al: Customizable natural language processing biomarker extraction tool. JCO Clin Cancer Inform 10.1200\/CCI.21.00017","journal-title":"JCO Clin Cancer Inform"},{"key":"e_1_3_5_20_2","article-title":"RUBY: Natural language processing of French electronic medical records for breast cancer research","author":"Schiappa R","unstructured":"Schiappa R, Contu S, Culie D, et al: RUBY: Natural language processing of French electronic medical records for breast cancer research. JCO Clin Cancer Inform 10.1200\/CCI.21.00199","journal-title":"JCO Clin Cancer Inform"},{"key":"e_1_3_5_21_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-019-0931-8"},{"key":"e_1_3_5_22_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12874-022-01583-z"},{"key":"e_1_3_5_23_2","article-title":"Natural language processing approaches to detect the timeline of metastatic recurrence of breast cancer","author":"Banerjee I","unstructured":"Banerjee I, Bozkurt S, Caswell-Jin JL, et al: Natural language processing approaches to detect the timeline of metastatic recurrence of breast cancer. JCO Clin Cancer Inform 10.1200\/CCI.19.00034","journal-title":"JCO Clin Cancer Inform"},{"key":"e_1_3_5_24_2","doi-asserted-by":"publisher","DOI":"10.1093\/aje\/kwt441"},{"key":"e_1_3_5_25_2","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocac040"},{"key":"e_1_3_5_26_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.jpainsymman.2020.06.010"},{"key":"e_1_3_5_27_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.artmed.2023.102625"},{"key":"e_1_3_5_28_2","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocae069"},{"key":"e_1_3_5_29_2","doi-asserted-by":"publisher","DOI":"10.1055\/s-0041-1731784"},{"key":"e_1_3_5_30_2","doi-asserted-by":"publisher","DOI":"10.1056\/AIdbp2300110"},{"key":"e_1_3_5_31_2","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01079-8"},{"key":"e_1_3_5_32_2","doi-asserted-by":"publisher","DOI":"10.2196\/58457"},{"key":"e_1_3_5_33_2","doi-asserted-by":"publisher","DOI":"10.1109\/JBHI.2017.2767063"},{"key":"e_1_3_5_34_2","doi-asserted-by":"publisher","DOI":"10.1016\/S0140-6736(16)31891-8"},{"key":"e_1_3_5_35_2","doi-asserted-by":"publisher","DOI":"10.1002\/cncr.33102"},{"key":"e_1_3_5_36_2","doi-asserted-by":"publisher","DOI":"10.3390\/cancers14184543"},{"key":"e_1_3_5_37_2","doi-asserted-by":"publisher","DOI":"10.2196\/55318"},{"key":"e_1_3_5_38_2","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocad256"}],"container-title":["JCO Clinical Cancer Informatics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/ascopubs.org\/doi\/pdfdirect\/10.1200\/CCI-25-00133","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,2]],"date-time":"2026-04-02T21:32:32Z","timestamp":1775165552000},"score":1,"resource":{"primary":{"URL":"https:\/\/ascopubs.org\/doi\/10.1200\/CCI-25-00133"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,1]]},"references-count":37,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2026,1]]}},"alternative-id":["10.1200\/CCI-25-00133"],"URL":"https:\/\/doi.org\/10.1200\/cci-25-00133","relation":{},"ISSN":["2473-4276"],"issn-type":[{"value":"2473-4276","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,1]]},"assertion":[{"value":"2025-05-06","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2025-11-27","order":1,"name":"revised","label":"Revised","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-01-14","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-02-18","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}],"article-number":"e2500133"}}