{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,5]],"date-time":"2026-03-05T13:23:20Z","timestamp":1772717000267,"version":"3.50.1"},"reference-count":36,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T00:00:00Z","timestamp":1772150400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Biology"],"abstract":"<jats:p>Background and Purpose: We present a pilot benchmark dataset of 378 preclinical histological samples for evaluating large language model (LLM) performance on multi-dimensional classification tasks. This dataset addresses the lack of standardized benchmarks for assessing LLMs in preclinical histopathology, encompassing species identification (mouse, rabbit, rat), organ recognition, staining methods, and preparation techniques. Methods: We evaluated the LLMs GPT-4.1, GPT-4o-mini, and Llama 3.2 on 378 histological samples across four classification dimensions: species identification (mouse, rabbit, rat), organ recognition (kidney, liver, prostate, spleen), staining method classification (H&amp;E, Elastica van Gieson, collagen, iron, IHC-elastin, MOVAT\u2019s pentachrome), and preparation type determination (frozen vs. paraffin-embedded). Performance was assessed using sensitivity and specificity metrics with confusion matrix analysis. Results: Model performance varied substantially across tasks and exhibited strong sensitivity to class imbalance. For preparation type classification, GPT-4.1 achieved the most balanced performance (50% frozen sensitivity, 85.7% paraffin sensitivity), while Llama 3.2 failed to recognize paraffin samples (0% sensitivity). In species classification, Llama 3.2 was the only model capable of identifying all three species (rabbit: 75% sensitivity, rat: 85.7% sensitivity) despite poor mouse recognition (0.3% sensitivity). GPT-4.1 achieved higher mouse sensitivity within this dataset (70.4% sensitivity) but failed with minority species. For staining classification, Llama 3.2 demonstrated highest overall performance, achieving &gt;88% sensitivity for most staining types, while GPT-4o-mini showed perfect H&amp;E recognition (100% sensitivity). Conclusions: Current LLMs demonstrate variable performance for histological classification with substantial sensitivity to class imbalance. While not suitable for standalone diagnostic use, they may serve as useful screening tools in research settings with appropriate human oversight.<\/jats:p>","DOI":"10.3390\/biology15050395","type":"journal-article","created":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T17:05:16Z","timestamp":1772211916000},"page":"395","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":0,"title":["Preclinical HistoBench: A Pilot Benchmark Dataset for Evaluating Large Language Models on Preclinical Histopathological Classification"],"prefix":"10.3390","volume":"15","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-5453-1267","authenticated-orcid":false,"given":"Avan","family":"Kader","sequence":"first","affiliation":[{"name":"Department of Diagnostic and Interventional Radiology, Technical University of Munich, Ismaninger Str. 22, 81675 Munich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-5355-9030","authenticated-orcid":false,"given":"Marie-Luise H. H.","family":"Ranner-Hafferl","sequence":"additional","affiliation":[{"name":"Department of Radiology, Charit\u00e9\u2013Universit\u00e4tsmedizin Berlin, Corporate Member of Freie Universit\u00e4t Berlin, Humboldt-Universit\u00e4t zu Berlin, Charit\u00e9platz 1, 10117 Berlin, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0009-0006-2423-5666","authenticated-orcid":false,"given":"Felix","family":"Reuter","sequence":"additional","affiliation":[{"name":"Department of Diagnostic and Interventional Radiology, Technical University of Munich, Ismaninger Str. 22, 81675 Munich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Miriam L.","family":"Fichtner","sequence":"additional","affiliation":[{"name":"Department of Experimental Neurology and Neurology, Charit\u00e9\u2013Universit\u00e4tsmedizin Berlin, Corporate Member of Freie Universit\u00e4t Berlin, Humboldt-Universit\u00e4t zu Berlin, Charit\u00e9platz 1, 10117 Berlin, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Marcus R.","family":"Makowski","sequence":"additional","affiliation":[{"name":"Department of Diagnostic and Interventional Radiology, Technical University of Munich, Ismaninger Str. 22, 81675 Munich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Keno K.","family":"Bressem","sequence":"additional","affiliation":[{"name":"Department of Diagnostic and Interventional Radiology, Technical University of Munich, Ismaninger Str. 22, 81675 Munich, Germany"},{"name":"Department of Cardiovascular Radiology and Nuclear Medicine, Technical University of Munich, Ismaninger Str. 22, 81675 Munich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Lisa C.","family":"Adams","sequence":"additional","affiliation":[{"name":"Department of Diagnostic and Interventional Radiology, Technical University of Munich, Ismaninger Str. 22, 81675 Munich, Germany"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2026,2,27]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"34","DOI":"10.1016\/j.csbj.2018.01.001","article-title":"Machine Learning Methods for Histopathological Image Analysis","volume":"16","author":"Komura","year":"2018","journal-title":"Comput. Struct. Biotechnol. J."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"1122","DOI":"10.1001\/jama.2015.1405","article-title":"Diagnostic concordance among pathologists interpreting breast biopsy specimens","volume":"313","author":"Elmore","year":"2015","journal-title":"JAMA"},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"649","DOI":"10.7326\/M15-0964","article-title":"Variability in Pathologists\u2019 Interpretations of Individual Breast Biopsy Slides: A Population Perspective","volume":"164","author":"Elmore","year":"2016","journal-title":"Ann. Intern. Med."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Niu, Y., Li, J., Xu, X., Luo, P., Liu, P., Wang, J., and Mu, J. (2024). Deep learning-driven ultrasound-assisted diagnosis: Optimizing GallScopeNet for precise identification of biliary atresia. Front. Med., 11.","DOI":"10.3389\/fmed.2024.1445069"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Reddy, S., Shaheed, A., Seo, Y., and Patel, R. (2024). Development of an Artificial Intelligence Model for the Classification of Gastric Carcinoma Stages Using Pathology Slides. Cureus, 16.","DOI":"10.7759\/cureus.56740"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"62","DOI":"10.1111\/joim.13030","article-title":"Artificial intelligence as the next step towards precision pathology","volume":"288","author":"Acs","year":"2020","journal-title":"J. Intern. Med."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"e288","DOI":"10.1016\/S2589-7500(23)00025-0","article-title":"The impact of commercial health datasets on medical research and health-care algorithms","volume":"5","author":"Alberto","year":"2023","journal-title":"Lancet Digit. Health"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Matsuzaka, Y., and Yashiro, R. (2025). The Diagnostic Classification of the Pathological Image Using Computer Vision. Algorithms, 18.","DOI":"10.3390\/a18020096"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"Urooj, B., Fayaz, M., Ali, S., Dang, L.M., and Kim, K.W. (2025). Large Language Models in Medical Image Analysis: A Systematic Survey and Future Directions. Bioengineering, 12.","DOI":"10.3390\/bioengineering12080818"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Gertz, R.J., Bunck, A.C., Lennartz, S., Dratsch, T., Iuga, A.I., Maintz, D., and Kottlors, J. (2023). GPT-4 for Automated Determination of Radiological Study and Protocol based on Radiology Request Forms: A Feasibility Study. Radiology, 307.","DOI":"10.1148\/radiol.230877"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"9505","DOI":"10.1007\/s00432-023-04824-w","article-title":"Large language models for oncological applications","volume":"149","author":"Sorin","year":"2023","journal-title":"J. Cancer Res. Clin. Oncol."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"990","DOI":"10.1016\/j.jacr.2023.05.003","article-title":"Evaluating GPT as an Adjunct for Radiologic Decision Making: GPT-4 Versus GPT-3.5 in a Breast Imaging Pilot","volume":"20","author":"Rao","year":"2023","journal-title":"J. Am. Coll. Radiol."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"357","DOI":"10.1038\/s41586-023-06160-y","article-title":"Health system-scale language models are all-purpose prediction engines","volume":"619","author":"Jiang","year":"2023","journal-title":"Nature"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"44","DOI":"10.1038\/s41523-023-00557-8","article-title":"Large language model (ChatGPT) as a support tool for breast tumor board","volume":"9","author":"Sorin","year":"2023","journal-title":"npj Breast Cancer"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"784","DOI":"10.1111\/his.13691","article-title":"Evaluating the benefits of digital pathology implementation: Time savings in laboratory logistics","volume":"73","author":"Baidoshvili","year":"2018","journal-title":"Histopathology"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"1301","DOI":"10.1038\/s41591-019-0508-1","article-title":"Clinical-grade computational pathology using weakly supervised deep learning on whole slide images","volume":"25","author":"Campanella","year":"2019","journal-title":"Nat. Med."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1550","DOI":"10.1038\/s41467-021-21735-x","article-title":"Whole-brain tissue mapping toolkit using large-scale highly multiplexed immunofluorescence imaging and deep neural networks","volume":"12","author":"Maric","year":"2021","journal-title":"Nat. Commun."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Kader, A., Kaufmann, J.O., Mangarova, D.B., Moeckel, J., Brangsch, J., Adams, L.C., Zhao, J., Reimann, C., Saatz, J., and Traub, H. (2022). Iron Oxide Nanoparticles for Visualization of Prostate Cancer in MRI. Cancers, 14.","DOI":"10.3390\/cancers14122909"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Keller, S., Borde, T., Brangsch, J., Adams, L.C., Kader, A., Reimann, C., Gebert, P., Hamm, B., and Makowski, M. (2020). Native T1 Mapping Magnetic Resonance Imaging as a Quantitative Biomarker for Characterization of the Extracellular Matrix in a Rabbit Hepatic Cancer Model. Biomedicines, 8.","DOI":"10.3390\/biomedicines8100412"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Kader, A., Kaufmann, J.O., Mangarova, D.B., Moeckel, J., Adams, L.C., Brangsch, J., Heyl, J.L., Zhao, J., Verlemann, C., and Karst, U. (2022). Collagen-Specific Molecular Magnetic Resonance Imaging of Prostate Cancer. Int. J. Mol. Sci., 24.","DOI":"10.3390\/ijms24010711"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"20785","DOI":"10.1038\/s41598-020-77624-8","article-title":"Assessment of the hepatic tumor extracellular matrix using elastin-specific molecular magnetic resonance imaging in an experimental rabbit cancer model","volume":"10","author":"Keller","year":"2020","journal-title":"Sci. Rep."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Kader, A., Brangsch, J., Reimann, C., Kaufmann, J.O., Mangarova, D.B., Moeckel, J., Adams, L.C., Zhao, J., Saatz, J., and Traub, H. (2021). Visualization and Quantification of the Extracellular Matrix in Prostate Cancer Using an Elastin Specific Molecular Probe. Biology, 10.","DOI":"10.3390\/biology10111217"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"213884","DOI":"10.1016\/j.bioadv.2024.213884","article-title":"Sensitivity of magnetic resonance elastography to extracellular matrix and cell motility in human prostate cancer cell line-derived xenograft models","volume":"161","author":"Kader","year":"2024","journal-title":"Biomater. Adv."},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"9999847","DOI":"10.1155\/2021\/9999847","article-title":"Effect of Doxycycline on Survival in Abdominal Aortic Aneurysms in a Mouse Model","volume":"2021","author":"Adams","year":"2021","journal-title":"Contrast Media Mol. Imaging"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"852","DOI":"10.1111\/cup.14481","article-title":"Artificial intelligence and frozen section histopathology: A systematic review","volume":"50","author":"Gorman","year":"2023","journal-title":"J. Cutan. Pathol."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1288","DOI":"10.1038\/s41374-020-0455-y","article-title":"Multiclass cancer classification in fresh frozen and formalin-fixed paraffin-embedded tissue by DigiWest multiplex protein analysis","volume":"100","author":"Bockmayr","year":"2020","journal-title":"Lab. Investig."},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Crowley, L., Cambuli, F., Aparicio, L., Shibata, M., Robinson, B.D., Xuan, S., Li, W., Hibshoosh, H., Loda, M., and Rabadan, R. (2020). A single-cell atlas of the mouse and human prostate reveals heterogeneity and conservation of epithelial progenitors. eLife, 9.","DOI":"10.7554\/eLife.59465"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"865","DOI":"10.1177\/0192623319873871","article-title":"Histology Atlas of the Developing Mouse Urinary System With Emphasis on Prenatal Days E10.5\u2013E18.5","volume":"47","author":"Elmore","year":"2019","journal-title":"Toxicol. Pathol."},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"455","DOI":"10.1007\/BF00301871","article-title":"Anatomy of Cecum of the Laboratory Mouse and Rat","volume":"162","author":"Snipes","year":"1981","journal-title":"Anat. Embryol."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Sanger, C., Schenk, A., Schwen, L.O., Wang, L., Gremse, F., Zafarnia, S., Kiessling, F., Xie, C., Wei, W., and Richter, B. (2015). Intrahepatic Vascular Anatomy in Rats and Mice\u2014Variations and Surgical Implications. PLoS ONE, 10.","DOI":"10.1371\/journal.pone.0141798"},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"437","DOI":"10.1111\/joa.12032","article-title":"Isolation, characterization and the multi-lineage differentiation potential of rabbit bone marrow-derived mesenchymal stem cells","volume":"222","author":"Tan","year":"2013","journal-title":"J. Anat."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"1697","DOI":"10.1007\/s00192-016-3008-6","article-title":"Comparative histology of mouse, rat, and human pelvic ligaments","volume":"27","author":"Iwanaga","year":"2016","journal-title":"Int. Urogynecol. J."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"7","DOI":"10.3109\/10520298409113823","article-title":"Identification in histological sections of species origin of cells from mouse, rat and human","volume":"59","author":"Cunha","year":"1984","journal-title":"Stain Technol."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1186\/s40537-019-0192-5","article-title":"Survey on deep learning with class imbalance","volume":"6","author":"Johnson","year":"2019","journal-title":"J. Big Data"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"10104","DOI":"10.1038\/s41467-024-51465-9","article-title":"In-context learning enables multimodal large language models to classify cancer pathology images","volume":"15","author":"Ferber","year":"2024","journal-title":"Nat. Commun."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Cheng, J. (2024). Applications of Large Language Models in Pathology. Bioengineering, 11.","DOI":"10.3390\/bioengineering11040342"}],"container-title":["Biology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2079-7737\/15\/5\/395\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,3,5]],"date-time":"2026-03-05T12:24:53Z","timestamp":1772713493000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2079-7737\/15\/5\/395"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,2,27]]},"references-count":36,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2026,3]]}},"alternative-id":["biology15050395"],"URL":"https:\/\/doi.org\/10.3390\/biology15050395","relation":{},"ISSN":["2079-7737"],"issn-type":[{"value":"2079-7737","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,2,27]]}}}