{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T21:37:55Z","timestamp":1792791475977,"version":"4.5.0"},"reference-count":40,"publisher":"JMIR Publications Inc.","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["JMIR Form Res"],"abstract":"Abstract<\/jats:title>\n \n Background<\/jats:title>\n Triple-negative breast cancer (TNBC) is an aggressive molecular subtype of breast cancer that needs to be identified accurately from pathology reports to inform decision-making. Large language models (LLMs) have been reported to show great potential in the automatic extraction of important oncology information from unstructured medical text.<\/jats:p>\n <\/jats:sec>\n \n Objective<\/jats:title>\n We aimed to create and test an LLM-based approach for automatic biomarker extraction and TNBC subtype classification based on breast cancer pathology reports using prompt-engineering techniques.<\/jats:p>\n <\/jats:sec>\n \n Methods<\/jats:title>\n In total, 532 breast cancer pathology reports from The Cancer Genome Atlas (TCGA) were used. Our approach extracted structured oncology features, such as estrogen receptor status, progesterone receptor status, HER2 status, tumor size, tumor grade, and tumor histology, which were used to identify TNBC. We tested 4 frontier models: GPT-4o mini (released July 18, 2024), GPT-4.1 nano (released April 14, 2025), GPT-5 nano (released August 7, 2025), and Gemini 2.5 Flash-Lite.<\/jats:p>\n <\/jats:sec>\n \n Results<\/jats:title>\n The designed framework was able to extract relevant biomarkers and clinicopathological attributes from unstructured pathology reports and allowed for the automated identification of patients with TNBC. Gemini 2.5 Flash-Lite showed superior performance in the extraction of clinicopathological attributes. For all models investigated, chain-of-thought (CoT) prompting was shown to outperform other strategies, such as zero-shot and few-shot prompting, for both biomarker extraction and TNBC classification. GPT-4o mini using CoT prompts reported the highest performance for TNBC classification.<\/jats:p>\n <\/jats:sec>\n \n Conclusions<\/jats:title>\n Using prompt-engineering techniques based on LLMs proved to be an efficient method of translating unstructured breast cancer pathology reports into structured clinical information. Moreover, it can be used to accurately identify the TNBC subtype.<\/jats:p>\n <\/jats:sec>","DOI":"10.2196\/97117","type":"journal-article","created":{"date-parts":[[2026,9,23]],"date-time":"2026-09-23T13:50:06Z","timestamp":1790171406000},"page":"e97117-e97117","source":"Crossref","is-referenced-by-count":0,"title":["Large Language Model Framework for Structured Biomarker Extraction and Triple-Negative Breast Cancer Identification From Pathology Reports: Development and Evaluation Study"],"prefix":"10.2196","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0009-0003-6368-741X","authenticated-orcid":false,"given":"Nirmit","family":"Jain","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-6017-3661","authenticated-orcid":false,"given":"Trisha","family":"Annambhotla","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0003-3443-1562","authenticated-orcid":false,"given":"Akshara","family":"Siddharthan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-8304-8414","authenticated-orcid":false,"given":"Harshini","family":"Kanala","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-2329-940X","authenticated-orcid":false,"given":"Armaan","family":"Budhiraja","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0005-0143-645X","authenticated-orcid":false,"given":"Manglam","family":"Goutam Sinha","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0007-3887-3199","authenticated-orcid":false,"given":"Devyani","family":"Charan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0009-0009-0037-9341","authenticated-orcid":false,"given":"Muzaffar","family":"Ahmad Dar","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7941-602X","authenticated-orcid":false,"given":"Trupti","family":"Patel","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6655-4450","authenticated-orcid":false,"given":"Biji","family":"C L","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1010","published-online":{"date-parts":[[2026,10,23]]},"reference":[{"issue":"3","key":"R1","doi-asserted-by":"publisher","first-page":"154","DOI":"10.1007\/s12609-014-0152-1","article-title":"Molecular heterogeneity of triple negative breast cancer","volume":"6","author":"Abramson","journal-title":"Curr Breast Cancer Rep"},{"issue":"Suppl 2","key":"R2","doi-asserted-by":"publisher","DOI":"10.4172\/2161-1041.S2-001","article-title":"Triple negative breast cancer - an overview","volume":"2013","author":"Aysola","journal-title":"Hereditary Genet"},{"issue":"15 Pt 1","key":"R3","doi-asserted-by":"publisher","first-page":"4429","DOI":"10.1158\/1078-0432.CCR-06-3045","article-title":"Triple-negative breast cancer: clinical features and patterns of recurrence","volume":"13","author":"Dent","journal-title":"Clin Cancer Res"},{"issue":"4","key":"R4","doi-asserted-by":"publisher","first-page":"195","DOI":"10.1200\/JOP.777003","article-title":"American Society of Clinical Oncology\/College of American Pathologists guideline recommendations for immunohistochemical testing of estrogen and progesterone receptors in breast cancer","volume":"6","author":"Hammond","journal-title":"J Oncol Pract"},{"key":"R5","unstructured":"Deka P . BioBERT-mnli-snli-scinli-scitail-mednli-stsb. Hugging Face. URL: https:\/\/huggingface.co\/pritamdeka\/BioBERT-mnli-snli-scinli-scitail-mednli-stsb [Accessed 30-09-2026]"},{"key":"R6","doi-asserted-by":"publisher","DOI":"10.1016\/j.jcpo.2025.100567","article-title":"Capturing breast cancer subtypes in cancer registries: insights into real-world incidence and survival","volume":"44","author":"Vaz-Gon\u00e7alves","journal-title":"J Cancer Policy"},{"issue":"1","key":"R7","doi-asserted-by":"publisher","first-page":"77","DOI":"10.5083\/ejcm\/25-01-13","article-title":"Molecular and histopathological correlation of breast cancer subtypes with prognostic markers in Eastern India: a study from a tertiary care center","volume":"15","author":"Ramchandwani","journal-title":"Eur J Cardiovasc Med"},{"issue":"1","key":"R8","doi-asserted-by":"publisher","DOI":"10.1186\/s13578-025-01359-0","article-title":"Classifications of triple-negative breast cancer: insights and current therapeutic approaches","volume":"15","author":"Chen","journal-title":"Cell Biosci"},{"issue":"3","key":"R9","doi-asserted-by":"publisher","DOI":"10.3390\/cells12030367","article-title":"Systemically identifying triple-negative breast cancer subtype-specific prognosis signatures, based on single-cell RNA-seq data","volume":"12","author":"Xing","journal-title":"Cells"},{"issue":"4","key":"R10","doi-asserted-by":"publisher","DOI":"10.1016\/j.esmoop.2024.102993","article-title":"Subtyping of triple-negative breast cancers: its prognostication and implications in diagnosis of breast origin","volume":"9","author":"Hu","journal-title":"ESMO Open"},{"issue":"5","key":"R11","doi-asserted-by":"publisher","DOI":"10.3390\/biomedicines13051100","article-title":"Clinical significance of tumor grade in triple-negative breast cancer: a retrospective cohort analysis","volume":"13","author":"Ramanan","journal-title":"Biomedicines"},{"issue":"7972","key":"R12","doi-asserted-by":"publisher","first-page":"172","DOI":"10.1038\/s41586-023-06291-2","article-title":"Large language models encode clinical knowledge","volume":"620","author":"Singhal","journal-title":"Nature"},{"issue":"3","key":"R13","doi-asserted-by":"publisher","first-page":"310","DOI":"10.1002\/path.6232","article-title":"Extracting structured information from unstructured histopathology reports using generative pre-trained transformer 4 (GPT-4)","volume":"262","author":"Truhn","journal-title":"J Pathol"},{"key":"R14","doi-asserted-by":"publisher","DOI":"10.2196\/70190","article-title":"Enhancing large language models for improved accuracy and safety in medical question answering: comparative study","volume":"11","author":"Wang","journal-title":"JMIR Med Educ"},{"issue":"3","key":"R15","doi-asserted-by":"publisher","first-page":"141","DOI":"10.1007\/s10032-009-0094-8","article-title":"Optical character recognition errors and their effects on natural language processing","volume":"12","author":"Lopresti","journal-title":"Int J Doc Anal Recognit"},{"key":"R16","doi-asserted-by":"crossref","unstructured":"Shuster K Chen M Kiela D Weston J . Retrieval augmentation reduces hallucination in conversation. arXiv. Preprint posted online on 2021. [doi: 10.48550\/arXiv.2104.07567]","DOI":"10.18653\/v1\/2021.findings-emnlp.320"},{"issue":"2","key":"R17","doi-asserted-by":"publisher","DOI":"10.1371\/journal.pdig.0000198","article-title":"Performance of ChatGPT on USMLE: potential for AI-assisted medical education using large language models","volume":"2","author":"Kung","journal-title":"PLOS Digit Health"},{"issue":"4","key":"R18","doi-asserted-by":"publisher","first-page":"2231","DOI":"10.7314\/apjcp.2013.14.4.2231","article-title":"Her-2\/neu status: a neglected marker of prognostication and management of breast cancer patients in India","volume":"14","author":"Zubeda","journal-title":"Asian Pac J Cancer Prev"},{"key":"R19","unstructured":"Wallace E Xiao K Leike RH Weng L Heidecke J Beutel A . The instruction hierarchy: training LLMs to prioritize privileged instructions. arXiv. Preprint posted online on 2024. [doi: 10.48550\/arXiv.2404.13208]"},{"key":"R20","doi-asserted-by":"crossref","unstructured":"Kojima T Gu S (Shane Reid M Matsuo Y Iwasawa Y . Large language models are zero-shot reasoners. Presented at: NIPS\u201922: Proceedings of the 36th International Conference on Neural Information Processing Systems; Nov 28 to Dec 9, 2022:22199-22213; New Orleans, Louisiana, USA. [doi: 10.52202\/068431-1613]","DOI":"10.52202\/068431-1613"},{"key":"R21","doi-asserted-by":"crossref","unstructured":"Wei J Wang X Schuurmans D et al. Chain-of-thought prompting elicits reasoning in large language models. Presented at: NIPS\u201922: Proceedings of the 36th International Conference on Neural Information Processing Systems; Nov 28 to Dec 9, 2022:24824-24837; New Orleans, Louisiana, USA. URL: http:\/\/www.proceedings.com\/68431.html [doi: 10.52202\/068431-1800]","DOI":"10.52202\/068431-1800"},{"issue":"1","key":"R22","doi-asserted-by":"publisher","DOI":"10.1038\/s41523-022-00468-0","article-title":"Triple negative breast cancer: pitfalls and progress","volume":"8","author":"Zagami","journal-title":"NPJ Breast Cancer"},{"key":"R23","doi-asserted-by":"publisher","DOI":"10.2147\/BCTT.S513779","article-title":"A machine-learning model for the prediction of triple-negative breast cancer based on multiparameter MRI","volume":"17","author":"Cai","journal-title":"Breast Cancer (Dove Med Press)"},{"key":"R24","doi-asserted-by":"publisher","DOI":"10.3389\/fmed.2025.1753443","article-title":"Exploring agentic AI in healthcare: a study on its working mechanism","volume":"12","author":"Srinivasu","journal-title":"Front Med (Lausanne)"},{"key":"R25","doi-asserted-by":"publisher","DOI":"10.3389\/frai.2025.1622292","article-title":"Survey and analysis of hallucinations in large language models: attribution to prompting strategies or model behavior","volume":"8","author":"Anh-Hoang","journal-title":"Front Artif Intell"},{"key":"R26","unstructured":"Wang X Wei J Schuurmans D Le Q Chi E Zhou D . Self-consistency improves chain of thought reasoning in language models. arXiv. Preprint posted online on 2022. [doi: 10.48550\/arXiv.2203.11171]"},{"key":"R27","doi-asserted-by":"crossref","unstructured":"Wei J Wang X Schuurmans D et al. Advances in neural information processing systems 35: 36th conference on neural information processing systems (neurips 2022). Preprint posted online on 2022. [doi: 10.52202\/068431-1800]","DOI":"10.52202\/068431-1800"},{"issue":"1","key":"R28","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-025-01686-z","article-title":"Iterative refinement and goal articulation to optimize large language models for clinical information extraction","volume":"8","author":"Hein","journal-title":"NPJ Digit Med"},{"key":"R29","doi-asserted-by":"publisher","DOI":"10.2196\/55318","article-title":"An empirical evaluation of prompting strategies for large language models in zero-shot clinical natural language processing: algorithm development and validation study","volume":"12","author":"Sivarajkumar","journal-title":"JMIR Med Inform"},{"key":"R30","doi-asserted-by":"publisher","DOI":"10.1200\/CCI-25-00133","article-title":"Development and assessment of a pipeline for extracting structured data from free-text medical reports using a large language model","volume":"10","author":"Joseph","journal-title":"JCO Clin Cancer Inform"},{"issue":"1","key":"R31","doi-asserted-by":"publisher","DOI":"10.1186\/s12911-025-02954-4","article-title":"A systematic review of large language model (LLM) evaluations in clinical medicine","volume":"25","author":"Shool","journal-title":"BMC Med Inform Decis Mak"},{"issue":"1","key":"R32","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-024-01219-0","article-title":"A scoping review of large language model based approaches for information extraction from radiology reports","volume":"7","author":"Reichenpfader","journal-title":"NPJ Digit Med"},{"issue":"3","key":"R33","doi-asserted-by":"publisher","first-page":"209","DOI":"10.3857\/roj.2023.00633","article-title":"Developing prompts from large language model for extracting clinical information from pathology and ultrasound reports in breast cancer","volume":"41","author":"Choi","journal-title":"Radiat Oncol J"},{"issue":"2","key":"R34","doi-asserted-by":"publisher","first-page":"210","DOI":"10.7326\/M23-2772","article-title":"Large language models in medicine: the potentials and pitfalls: a narrative review","volume":"177","author":"Omiye","journal-title":"Ann Intern Med"},{"issue":"1","key":"R35","doi-asserted-by":"publisher","DOI":"10.1038\/s41746-023-00879-8","article-title":"The shaky foundations of large language models and foundation models for electronic health records","volume":"6","author":"Wornow","journal-title":"NPJ Digit Med"},{"key":"R36","unstructured":"Model context protocol. Anthropic. URL: https:\/\/www.anthropic.com\/news\/model-context-protocol [Accessed 30-09-2026]"},{"issue":"1","key":"R37","doi-asserted-by":"publisher","first-page":"6","DOI":"10.1007\/s44250-026-00343-w","article-title":"Transforming clinical medicine with multimodal artificial intelligence, agentic systems, and the model-context protocol: a perspective on future directions","volume":"5","author":"Choi","journal-title":"Discov Health Syst"},{"key":"R38","doi-asserted-by":"crossref","unstructured":"Zeng KG Dutt T Witowski J et al. Improving information extraction from pathology reports using named entity recognition. Res Sq. Jul 3, 2023:rs.3.rs-3035772. [doi: 10.21203\/rs.3.rs-3035772\/v1] [Medline: 37461545]","DOI":"10.21203\/rs.3.rs-3035772\/v1"},{"key":"R39","doi-asserted-by":"publisher","DOI":"10.1016\/j.jbi.2015.07.010","article-title":"Challenges in clinical natural language processing for automated disorder normalization","volume":"57","author":"Leaman","journal-title":"J Biomed Inform"},{"key":"R40","unstructured":"Nirmitjain2110\/large-language-model-framework-for-structured-biomarker-extraction-and-triplenegative-breast-cancer. GitHub. URL: https:\/\/github.com\/nirmitjain2110\/Large-Language-Model-Framework-for-Structured-Biomarker-Extraction-and-TripleNegative-Breast-Cancer [Accessed 01-10-2026]"}],"container-title":["JMIR Formative Research"],"original-title":[],"language":"en","deposited":{"date-parts":[[2026,10,23]],"date-time":"2026-10-23T20:43:38Z","timestamp":1792788218000},"score":1,"resource":{"primary":{"URL":"https:\/\/formative.jmir.org\/2026\/1\/e97117"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,10,23]]},"references-count":40,"URL":"https:\/\/doi.org\/10.2196\/97117","relation":{},"ISSN":["2561-326X"],"issn-type":[{"value":"2561-326X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,10,23]]},"article-number":"v10i3e97117"}}