{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,9]],"date-time":"2026-10-09T17:29:16Z","timestamp":1791566956548,"version":"4.3.4"},"reference-count":25,"publisher":"Springer Science and Business Media LLC","license":[{"start":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T00:00:00Z","timestamp":1776124800000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T00:00:00Z","timestamp":1776124800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100000070","name":"National Institute of Biomedical Imaging and Bioengineering","doi-asserted-by":"publisher","award":["R01EB033782"],"award-info":[{"award-number":["R01EB033782"]}],"id":[{"id":"10.13039\/100000070","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["J Digit Imaging. Inform. med."],"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:p>\n                    Vision-language models can connect the text description of an object to its specific location in an image through visual grounding. This has potential applications in enhanced radiology reporting. However, these models require large annotated image-text datasets, which are lacking for PET\/CT. We developed an automated pipeline to generate weak image-text labels and used it to train a 3D visual grounding model. Our weak-labeling pipeline identified sentences describing positive findings in PET\/CT reports by searching for mentions of standardized uptake values (SUVmax) and axial slice numbers. These were used to automatically generate lesion masks, which were paired with the corresponding text descriptions. From 25,578 PET\/CT exams, we extracted 11,356 sentence-label pairs. Using this data, we trained ConTEXTual Net 3D, which takes as input a description of a lesion and generates a corresponding segmentation mask. The model\u2019s performance was evaluated on 251 radiologist-reviewed cases and compared against LLMSeg, a 2.5D version of ConTEXTual Net, and two radiologists. We evaluated detection performance using F1 score. The weak-labeling pipeline accurately identified lesion locations in 98% of cases (246\/251). ConTEXTual Net 3D achieved an F1 score of 0.80, outperforming LLMSeg (F1\u2009=\u20090.22) and the 2.5D model (F1\u2009=\u20090.53), though it underperformed both radiologists (F1\u2009=\u20090.94 and 0.91). The model achieved better performance on\n                    <jats:sup>18<\/jats:sup>\n                    F-fluorodeoxyglucose (F1\u2009=\u20090.78) and DCFPyL (F1\u2009=\u20090.75) exams than on DOTATATE (F1\u2009=\u20090.58) and\n                    <jats:sup>18<\/jats:sup>\n                    F-fluciclovine (F1\u2009=\u20090.66) exams. In conclusion,\u00a0our novel weak labeling pipeline accurately produced an annotated dataset of PET\/CT image-text pairs. ConTEXTual Net 3D significantly outperformed other models but fell short of the performance of nuclear medicine physicians. Our study suggests that even larger datasets may be needed to close this performance gap.\n                  <\/jats:p>","DOI":"10.1007\/s10278-026-01879-2","type":"journal-article","created":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T13:01:31Z","timestamp":1776171691000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":4,"title":["ConTEXTual Net 3D: Vision-Language Modeling in PET\/CT for Visual Grounding of Positive Findings"],"prefix":"10.1007","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1472-243X","authenticated-orcid":false,"given":"Zachary","family":"Huemann","sequence":"first","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Samuel","family":"Church","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Joshua D.","family":"Warner","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel","family":"Tran","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xin","family":"Tie","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Alan B.","family":"McMillan","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Junjie","family":"Hu","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Steve Y.","family":"Cho","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Meghan G.","family":"Lubner","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Tyler J.","family":"Bradshaw","sequence":"additional","affiliation":[],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"297","published-online":{"date-parts":[[2026,4,14]]},"reference":[{"key":"1879_CR1","doi-asserted-by":"crossref","unstructured":"Li C, Wong C, Zhang S, et al. LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Day. June 2023.","DOI":"10.32388\/VLXB6M"},{"key":"1879_CR2","unstructured":"Pellegrini C, \u00d6zsoy E, Busam B, Navab N, Keicher M. RaDialog: A Large Vision-Language Model for Radiology Report Generation and Conversational Assistance. November 2023."},{"key":"1879_CR3","doi-asserted-by":"crossref","unstructured":"Folio LR, Machado LB, Dwyer AJ. Multimedia-enhanced Radiology Reports: Concept, Components, and Challenges. RadioGraphics. 2018;38:462-482.","DOI":"10.1148\/rg.2017170047"},{"key":"1879_CR4","doi-asserted-by":"crossref","unstructured":"Lee YH, Yang J, Suh J-S. Detection and Correction of Laterality Errors in Radiology Reports. J Digit Imaging. 2015;28:412-416.","DOI":"10.1007\/s10278-015-9772-x"},{"key":"1879_CR5","doi-asserted-by":"crossref","unstructured":"Vosshenrich J, Nesic I, Cyriac J, Boll DT, Merkle EM, Heye T. Revealing the most common reporting errors through data mining of the report proofreading process. Eur Radiol. 2021;31:2115-2125.","DOI":"10.1007\/s00330-020-07306-6"},{"key":"1879_CR6","doi-asserted-by":"crossref","unstructured":"Huemann Z, Tie X, Hu J, Bradshaw TJ. ConTEXTual Net: A Multimodal Vision-Language Model for Segmentation of Pneumothorax. J Digit Imaging Inform med. 2024;37:1652-1663.","DOI":"10.1007\/s10278-024-01051-8"},{"key":"1879_CR7","doi-asserted-by":"crossref","unstructured":"Zhao W, Li F, Diao Y, Fan P, Chen Z. Cap2Seg: leveraging caption generation for enhanced segmentation of COVID-19 medical images. Front Phys. 2024;12:1439122.","DOI":"10.3389\/fphy.2024.1439122"},{"key":"1879_CR8","doi-asserted-by":"crossref","unstructured":"Bannur S, Bouzid K, Castro DC, et al. MAIRA-2: Grounded Radiology Report Generation. September 2024.","DOI":"10.1056\/AIdbp2401120"},{"key":"1879_CR9","doi-asserted-by":"crossref","unstructured":"Oh Y, Park S, Byun HK, et al. LLM-driven multimodal target volume contouring in radiation oncology. Nat Commun. 2024;15:9186.","DOI":"10.1038\/s41467-024-53387-y"},{"key":"1879_CR10","unstructured":"Rajendran P, Chen Y, Qiu L, et al. Autodelineation of Treatment Target Volume for Radiation Therapy Using Large Language Model-Aided Multimodal Learning. International Journal of Radiation Oncology*Biology*Physics. August 2024:S0360301624029717."},{"key":"1879_CR11","unstructured":"McMillan AB, Li X, Bruce R, Garrett J, Bradshaw T. Enhancing Medical Imaging Interpretation Through Visual Grounding in Language-Vision Models."},{"key":"1879_CR12","unstructured":"Jain S, Agrawal A, Saporta A, et al. RadGraph: Extracting Clinical Entities and Relations from Radiology Reports. August 2021."},{"key":"1879_CR13","unstructured":"Jiang AQ, Sablayrolles A, Mensch A, et al. Mistral 7B. October 2023."},{"key":"1879_CR14","unstructured":"Jiang AQ, Sablayrolles A, Roux A, et al. Mixtral of Experts. January 2024."},{"key":"1879_CR15","unstructured":"Mukherjee S, Mitra A, Jawahar G, Agarwal S, Palangi H, Awadallah A. Orca: Progressive Learning from Complex Explanation Traces of GPT-4. June 2023."},{"key":"1879_CR16","doi-asserted-by":"crossref","unstructured":"Jentzen W. An Improved Iterative Thresholding Method to Delineate PET Volumes Using the Delineation-Averaged Signal Instead of the Enclosed Maximum Signal. Journal of Nuclear Medicine Technology. 2015;43:28-35.","DOI":"10.2967\/jnmt.114.152678"},{"key":"1879_CR17","doi-asserted-by":"crossref","unstructured":"Weisman AJ, Kieler MW, Perlman S, et al. Comparison of 11 automated PET segmentation methods in lymphoma. Phys Med Biol. 2020;65:235019.","DOI":"10.1088\/1361-6560\/abb6bd"},{"key":"1879_CR18","doi-asserted-by":"crossref","unstructured":"Isensee F, Jaeger PF, Kohl SAA, Petersen J, Maier-Hein KH. nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation. Nature Methods. 2021;18:203-211.","DOI":"10.1038\/s41592-020-01008-z"},{"key":"1879_CR19","unstructured":"Cardoso MJ, Li W, Brown R, et al. MONAI: An open-source framework for deep learning in healthcare. November 2022."},{"key":"1879_CR20","doi-asserted-by":"crossref","unstructured":"Kirillov A, Mintun E, Ravi N, et al. Segment Anything. April 2023.","DOI":"10.1109\/ICCV51070.2023.00371"},{"key":"1879_CR21","unstructured":"Liu Y, Ott M, Goyal N, et al. RoBERTa: A Robustly Optimized BERT Pretraining Approach. July 2019."},{"key":"1879_CR22","doi-asserted-by":"crossref","unstructured":"Yan A, McAuley J, Lu X, et al. RadBERT: Adapting Transformer-based Language Models to Radiology. Radiology: Artificial Intelligence. 2022;4:e210258.","DOI":"10.1148\/ryai.210258"},{"key":"1879_CR23","unstructured":"Devlin J, Chang M-W, Lee K, Toutanova K. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding."},{"key":"1879_CR24","unstructured":"Grattafiori A, Dubey A, Jauhri A, et al. The Llama 3 Herd of Models. November 2024."},{"key":"1879_CR25","unstructured":"Ahamed S, Xu Y, Kurkowska S, et al. Comprehensive framework for evaluation of deep neural networks in detection and quantification of lymphoma from PET\/CT images: clinical insights, pitfalls, and observer agreement analyses. December 2024."}],"container-title":["Journal of Imaging Informatics in Medicine"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10278-026-01879-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10278-026-01879-2","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10278-026-01879-2.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,4,14]],"date-time":"2026-04-14T13:01:41Z","timestamp":1776171701000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10278-026-01879-2"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,14]]},"references-count":25,"alternative-id":["1879"],"URL":"https:\/\/doi.org\/10.1007\/s10278-026-01879-2","relation":{},"ISSN":["2948-2933"],"issn-type":[{"value":"2948-2933","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,4,14]]},"assertion":[{"value":"21 September 2025","order":1,"name":"received","label":"Received","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"1 February 2026","order":2,"name":"revised","label":"Revised","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"10 February 2026","order":3,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 April 2026","order":4,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"Research reported in this publication was supported by the National Institute of Biomedical Imaging and Bioengineering of the National Institutes of Health under Award No. R01EB033782. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.\u00a0Zachary Huemann received Nvidia RTX6000 GPU as an Academic Hardware Grant in support of this project.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"Competing interests"}}]}}