{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T01:00:43Z","timestamp":1790211643591,"version":"4.1.0"},"reference-count":68,"publisher":"SPIE-Intl Soc Optical Eng","issue":"02","content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Med. Imag."],"published-print":{"date-parts":[[2024,4,8]]},"abstract":"<jats:p>PurposeSegmentation of the prostate and surrounding organs at risk from computed tomography is required for radiation therapy treatment planning. We propose an automatic two-step deep learning-based segmentation pipeline that consists of an initial multi-organ segmentation network for organ localization followed by organ-specific fine segmentation.ApproachInitial segmentation of all target organs is performed using a hybrid convolutional-transformer model, axial cross-attention UNet. The output from this model allows for region of interest computation and is used to crop tightly around individual organs for organ-specific fine segmentation. Information from this network is also propagated to the fine segmentation stage through an image enhancement module, highlighting regions of interest in the original image that might be difficult to segment. Organ-specific fine segmentation is performed on these cropped and enhanced images to produce the final output segmentation.ResultsWe apply the proposed approach to segment the prostate, bladder, rectum, seminal vesicles, and femoral heads from male pelvic computed tomography (CT). When tested on a held-out test set of 30 images, our two-step pipeline outperformed other deep learning-based multi-organ segmentation algorithms, achieving average dice similarity coefficient (DSC) of 0.836\u00b10.071 (prostate), 0.947\u00b10.038 (bladder), 0.828\u00b10.057 (rectum), 0.724\u00b10.101 (seminal vesicles), and 0.933\u00b10.020 (femoral heads).ConclusionsOur results demonstrate that a two-step segmentation pipeline with initial multi-organ segmentation and additional fine segmentation can delineate male pelvic CT organs well. The utility of this additional layer of fine segmentation is most noticeable in challenging cases, as our two-step pipeline produces noticeably more accurate and less erroneous results compared to other state-of-the-art methods on such images.<\/jats:p>","DOI":"10.1117\/1.jmi.11.2.024009","type":"journal-article","created":{"date-parts":[[2024,4,8]],"date-time":"2024-04-08T15:54:47Z","timestamp":1712591687000},"page":"1-17","source":"Crossref","is-referenced-by-count":3,"title":["Cascaded cross-attention transformers and convolutional neural networks for multi-organ segmentation in male pelvic computed tomography"],"prefix":"10.1117","volume":"11","author":[{"given":"Rahul","family":"Pemmaraju","sequence":"first","affiliation":[{"name":"Johns Hopkins Univ."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0730-9169","authenticated-orcid":false,"given":"Gayoung","family":"Kim","sequence":"additional","affiliation":[{"name":"Johns Hopkins Univ."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lina","family":"Mekki","sequence":"additional","affiliation":[{"name":"Johns Hopkins Univ."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Daniel  Y.","family":"Song","sequence":"additional","affiliation":[{"name":"Johns Hopkins Univ."}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-3004-8625","authenticated-orcid":false,"given":"Junghoon","family":"Lee","sequence":"additional","affiliation":[{"name":"Johns Hopkins Univ."}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"189","reference":[{"key":"r1","doi-asserted-by":"publisher","DOI":"10.1016\/j.eururo.2011.09.027"},{"key":"r2","doi-asserted-by":"publisher","DOI":"10.1120\/jacmp.v2i2.2614"},{"key":"r3","doi-asserted-by":"publisher","DOI":"10.1007\/s00066-012-0117-0"},{"key":"r4","doi-asserted-by":"publisher","DOI":"10.1259\/bjr.20160406"},{"key":"r5","doi-asserted-by":"publisher","DOI":"10.1016\/S0167-8140(98)00021-8"},{"key":"r6","doi-asserted-by":"publisher","DOI":"10.1016\/S0360-3016(02)02950-4"},{"key":"r7","doi-asserted-by":"publisher","DOI":"10.1109\/ISBI.2011.5872795"},{"key":"r8","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-15989-3_6"},{"key":"r9","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-75757-3_31"},{"key":"r10","doi-asserted-by":"publisher","DOI":"10.1088\/0031-9155\/59\/6\/1471"},{"key":"r11","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2015.06.007"},{"key":"r12","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2016.2519264"},{"key":"r13","first-page":"234","article-title":"U-Net: convolutional networks for biomedical image segmentation","author":"Ronneberger","year":"2015"},{"key":"r14","article-title":"U-Net and its variants for medical image segmentation: theory and applications","author":"Siddique","year":"2020"},{"key":"r15","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6560\/aaf11c"},{"key":"r16","doi-asserted-by":"publisher","DOI":"10.1117\/1.JMI.7.5.055001"},{"key":"r17","doi-asserted-by":"publisher","DOI":"10.3390\/diagnostics10110959"},{"key":"r18","doi-asserted-by":"publisher","DOI":"10.1186\/s13014-021-01867-6"},{"key":"r19","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6560\/abf2f9"},{"key":"r20","doi-asserted-by":"publisher","DOI":"10.1016\/j.radonc.2019.09.028"},{"key":"r21","doi-asserted-by":"publisher","DOI":"10.1002\/mp.14580"},{"key":"r22","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2019.03.003"},{"key":"r23","doi-asserted-by":"publisher","DOI":"10.1016\/j.ijrobp.2019.03.017"},{"key":"r24","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2017.634"},{"key":"r25","doi-asserted-by":"publisher","DOI":"10.3389\/fcomp.2021.613981"},{"key":"r26","first-page":"2223","article-title":"Unpaired image-to-image translation using cycle-consistent adversarial networks","author":"Zhu","year":"2017"},{"key":"r27","first-page":"139","article-title":"Generative adversarial networks","author":"Goodfellow","year":"2014"},{"key":"r28","doi-asserted-by":"publisher","DOI":"10.1109\/TBME.2021.3117407"},{"key":"r29","first-page":"623","article-title":"The domain shift problem of medical image segmentation and vendor-adaptation by Unet-GAN","author":"Yan","year":"2019"},{"key":"r30","article-title":"An image is worth 16x16 words: transformers for image recognition at scale","author":"Dosovitskiy","year":"2020"},{"key":"r31","first-page":"3965","article-title":"CoAtNet: marrying convolution and attention for all data sizes","author":"Dai","year":"2021"},{"key":"r32","article-title":"Attention is all you need","author":"Vaswani","year":"2017"},{"key":"r33","doi-asserted-by":"publisher","DOI":"10.1145\/3505244"},{"key":"r34","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2022.3152247"},{"key":"r35","doi-asserted-by":"publisher","DOI":"10.1007\/s41095-022-0271-y"},{"key":"r36","first-page":"10012","article-title":"Swin transformer: hierarchical vision transformer using shifted windows","author":"Liu","year":"2021"},{"key":"r37","first-page":"10347","article-title":"Training data-efficient image transformers &amp; distillation through attention","author":"Touvron","year":"2020"},{"key":"r38","first-page":"213","article-title":"End-to-end object detection with transformers","author":"Carion","year":"2020"},{"key":"r39","article-title":"Deformable DETR: deformable transformers for end-to-end object detection","author":"Zhu","year":"2020"},{"key":"r40","first-page":"7262","article-title":"Segmenter: transformer for semantic segmentation","author":"Strudel","year":"2021"},{"key":"r41","first-page":"108","article-title":"Axial-DeepLab: stand-alone axial-attention for panoptic segmentation","author":"Wang","year":"2020"},{"key":"r42","first-page":"6881","article-title":"Rethinking semantic segmentation from a sequence-to-sequence perspective with transformers","author":"Zheng","year":"2021"},{"key":"r43","first-page":"574","article-title":"UNETR: transformers for 3D medical image segmentation","author":"Hatamizadeh","year":"2022"},{"key":"r44","article-title":"TransUNet: transformers make strong encoders for medical image segmentation","author":"Chen","year":"2021"},{"key":"r45","first-page":"171","article-title":"CoTr: efficiently bridging CNN and transformer for 3D medical image segmentation","author":"Xie","year":"2021"},{"key":"r46","first-page":"272","article-title":"Swin UNETR: swin transformers for semantic segmentation of brain tumors in MRI images","author":"Hatamizadeh","year":"2021"},{"key":"r47","first-page":"36","article-title":"Medical transformer: gated axial-attention for medical image segmentation","author":"Valanarasu","year":"2021"},{"key":"r48","first-page":"357","article-title":"CrossViT: cross-attention multi-scale vision transformer for image classification","author":"Chen","year":"2021"},{"key":"r49","article-title":"Perceiver IO: a general architecture for structured inputs & outputs","author":"Jaegle","year":"2021"},{"key":"r50","first-page":"5652","article-title":"SelfDoc: self-supervised document representation learning","author":"Li","year":"2021"},{"key":"r51","first-page":"267","article-title":"U-Net transformer: self and cross attention for medical image segmentation","author":"Petit","year":"2021"},{"key":"r52","doi-asserted-by":"publisher","DOI":"10.1117\/12.2653387"},{"key":"r53","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2020.3000314"},{"key":"r54","doi-asserted-by":"publisher","DOI":"10.1038\/s41592-020-01008-z"},{"key":"r55","article-title":"Instance normalization: the missing ingredient for fast stylization","author":"Ulyanov","year":"2017"},{"key":"r56","first-page":"1929","article-title":"Dropout: a simple way to prevent neural networks from overfitting","author":"Srivastava","year":"2014"},{"key":"r57","first-page":"3","article-title":"Rectifier nonlinearities improve neural network acoustic models","author":"Maas","year":"2013"},{"key":"r58","first-page":"448","article-title":"Batch normalization: accelerating deep network training by reducing internal covariate shift","author":"Ioffe","year":"2015"},{"key":"r59","article-title":"Understanding softmax confidence and uncertainty","author":"Pearce","year":"2021"},{"key":"r60","first-page":"427","article-title":"Deep neural networks are easily fooled: high confidence predictions for unrecognizable images","author":"Nguyen","year":"2015"},{"key":"r61","first-page":"770","article-title":"Deep residual learning for image recognition","author":"He","year":"2022"},{"key":"r62","first-page":"4700","article-title":"Densely connected convolutional networks","author":"Huang","year":"2017"},{"key":"r63","first-page":"565","article-title":"V-Net: fully convolutional neural networks for volumetric medical image segmentation","author":"Milletari","year":"2016"},{"key":"r64","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-67558-9_28"},{"key":"r65","article-title":"Mean teachers are better role models: weight-averaged consistency targets improve semi-supervised deep learning results","author":"Tarvainen","year":"2017"},{"key":"r66","article-title":"Monai: An open-source framework for deep learning in healthcare","author":"Cardoso","year":"2022"},{"key":"r67","article-title":"Decoupled weight decay regularization","author":"Loshchilov","year":"2019"},{"key":"r68","article-title":"nnFormer: interleaved transformer for volumetric segmentation","author":"Zhou","year":"2022"}],"container-title":["Journal of Medical Imaging"],"original-title":[],"link":[{"URL":"https:\/\/www.spiedigitallibrary.org\/journals\/journal-of-medical-imaging\/volume-11\/issue-02\/024009\/Cascaded-cross-attention-transformers-and-convolutional-neural-networks-for-multi\/10.1117\/1.JMI.11.2.024009.full","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,9,24]],"date-time":"2026-09-24T00:04:24Z","timestamp":1790208264000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.spiedigitallibrary.org\/journals\/journal-of-medical-imaging\/volume-11\/issue-02\/024009\/Cascaded-cross-attention-transformers-and-convolutional-neural-networks-for-multi\/10.1117\/1.JMI.11.2.024009.full"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,8]]},"references-count":68,"journal-issue":{"issue":"02","published-online":{"date-parts":[[2024,3,1]]}},"URL":"https:\/\/doi.org\/10.1117\/1.jmi.11.2.024009","relation":{},"ISSN":["2329-4302","2329-4310"],"issn-type":[{"value":"2329-4302","type":"print"},{"value":"2329-4310","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,4,8]]}}}