{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,6]],"date-time":"2026-10-06T14:26:15Z","timestamp":1791296775805,"version":"4.1.0"},"reference-count":88,"publisher":"Wiley","issue":"3","license":[{"start":{"date-parts":[[2023,9,30]],"date-time":"2023-09-30T00:00:00Z","timestamp":1696032000000},"content-version":"vor","delay-in-days":0,"URL":"http:\/\/onlinelibrary.wiley.com\/termsAndConditions#vor"}],"funder":[{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["C06 CA059267"],"award-info":[{"award-number":["C06 CA059267"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/100000002","name":"National Institutes of Health","doi-asserted-by":"publisher","award":["R01AG078250"],"award-info":[{"award-number":["R01AG078250"]}],"id":[{"id":"10.13039\/100000002","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["aapm.onlinelibrary.wiley.com"],"crossmark-restriction":true},"short-container-title":["Medical Physics"],"published-print":{"date-parts":[[2024,3]]},"abstract":"<jats:title>Abstract<\/jats:title>\n                  <jats:sec>\n                    <jats:title>Background<\/jats:title>\n                    <jats:p>Radiotherapy (RT) combined with cetuximab is the standard treatment for patients with inoperable head and neck cancers. Segmentation of head and neck (H&amp;N) tumors is a prerequisite for radiotherapy planning but a time\u2010consuming process. In recent years, deep convolutional neural networks (DCNN) have become the de facto standard for automated image segmentation. However, due to the expensive computational cost associated with enlarging the field of view in DCNNs, their ability to model long\u2010range dependency is still limited, and this can result in sub\u2010optimal segmentation performance for objects with background context spanning over long distances. On the other hand, Transformer models have demonstrated excellent capabilities in capturing such long\u2010range information in several semantic segmentation tasks performed on medical images.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Purpose<\/jats:title>\n                    <jats:p>Despite the impressive representation capacity of vision transformer models, current vision transformer\u2010based segmentation models still suffer from inconsistent and incorrect dense predictions when fed with multi\u2010modal input data. We suspect that the power of their self\u2010attention mechanism may be limited in extracting the complementary information that exists in multi\u2010modal data. To this end, we propose a novel segmentation model, debuted, Cross\u2010modal Swin Transformer (SwinCross), with cross\u2010modal attention (CMA) module to incorporate cross\u2010modal feature extraction at multiple resolutions.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Methods<\/jats:title>\n                    <jats:p>We propose a novel architecture for cross\u2010modal 3D semantic segmentation with two main components: (1) a cross\u2010modal 3D Swin Transformer for integrating information from multiple modalities (PET and CT), and (2) a cross\u2010modal shifted window attention block for learning complementary information from the modalities. To evaluate the efficacy of our approach, we conducted experiments and ablation studies on the HECKTOR 2021 challenge dataset. We compared our method against nnU\u2010Net (the backbone of the top\u20105 methods in HECKTOR 2021) and other state\u2010of\u2010the\u2010art transformer\u2010based models, including UNETR and Swin UNETR. The experiments employed a five\u2010fold cross\u2010validation setup using PET and CT images.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Results<\/jats:title>\n                    <jats:p>Empirical evidence demonstrates that our proposed method consistently outperforms the comparative techniques. This success can be attributed to the CMA module's capacity to enhance inter\u2010modality feature representations between PET and CT during head\u2010and\u2010neck tumor segmentation. Notably, SwinCross consistently surpasses Swin UNETR across all five folds, showcasing its proficiency in learning multi\u2010modal feature representations at varying resolutions through the cross\u2010modal attention modules.<\/jats:p>\n                  <\/jats:sec>\n                  <jats:sec>\n                    <jats:title>Conclusions<\/jats:title>\n                    <jats:p>\n                      We introduced a cross\u2010modal Swin Transformer for automating the delineation of head and neck tumors in PET and CT images. Our model incorporates a cross\u2010modality attention module, enabling the exchange of features between modalities at multiple resolutions. The experimental results establish the superiority of our method in capturing improved inter\u2010modality correlations between PET and CT for head\u2010and\u2010neck tumor segmentation. Furthermore, the proposed methodology holds applicability to other semantic segmentation tasks involving different imaging modalities like SPECT\/CT or PET\/MRI. Code:\n                      <jats:ext-link xmlns:xlink=\"http:\/\/www.w3.org\/1999\/xlink\" xlink:href=\"https:\/\/github.com\/yli192\/SwinCross_CrossModalSwinTransformer_for_Medical_Image_Segmentation\">https:\/\/github.com\/yli192\/SwinCross_CrossModalSwinTransformer_for_Medical_Image_Segmentation<\/jats:ext-link>\n                    <\/jats:p>\n                  <\/jats:sec>","DOI":"10.1002\/mp.16703","type":"journal-article","created":{"date-parts":[[2023,9,30]],"date-time":"2023-09-30T08:20:25Z","timestamp":1696062025000},"page":"2096-2107","update-policy":"https:\/\/doi.org\/10.1002\/crossmark_policy","source":"Crossref","is-referenced-by-count":32,"title":["SwinCross: Cross\u2010modal Swin transformer for head\u2010and\u2010neck tumor segmentation in PET\/CT images"],"prefix":"10.1002","volume":"51","author":[{"given":"Gary Y.","family":"Li","sequence":"first","affiliation":[{"name":"Center for Advanced Medical Computing and Analysis Massachusetts General Hospital\/Harvard Medical School  Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Junyu","family":"Chen","sequence":"additional","affiliation":[{"name":"The Russell H Morgan Department of Radiology and Radiological Science School of Medicine Johns Hopkins University  Baltimore Maryland USA"},{"name":"Department of Electrical and Computer Engineering Whiting School of Engineering Johns Hopkins University  Baltimore Maryland USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Se\u2010In","family":"Jang","sequence":"additional","affiliation":[{"name":"Center for Advanced Medical Computing and Analysis Massachusetts General Hospital\/Harvard Medical School  Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Kuang","family":"Gong","sequence":"additional","affiliation":[{"name":"Center for Advanced Medical Computing and Analysis Massachusetts General Hospital\/Harvard Medical School  Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Quanzheng","family":"Li","sequence":"additional","affiliation":[{"name":"Center for Advanced Medical Computing and Analysis Massachusetts General Hospital\/Harvard Medical School  Boston Massachusetts USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"311","published-online":{"date-parts":[[2023,9,30]]},"reference":[{"key":"e_1_2_8_2_1","doi-asserted-by":"publisher","DOI":"10.3322\/canjclin.55.2.74"},{"key":"e_1_2_8_3_1","volume-title":"Overview of the HECKTOR Challenge at MICCAI 2020: Automatic Head and Neck Tumor Segmentation in PET\/CT","author":"Andrearczyk V","year":"2021"},{"key":"e_1_2_8_4_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2023.102762"},{"key":"e_1_2_8_5_1","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TPAMI.2022.3206108","article-title":"VOLO: vision Outlooker for Visual Recognition","author":"Yuan Li","year":"2022","journal-title":"IEEE Trans Pattern Anal Mach Intell"},{"key":"e_1_2_8_6_1","doi-asserted-by":"crossref","unstructured":"YangC WangY ZhangJ et\u00a0al.Lite Vision Transformer with enhanced self\u2010attention.IEEE\/CVF Conference on Computer Vision and Pattern Recognition.2022.","DOI":"10.1109\/CVPR52688.2022.01169"},{"key":"e_1_2_8_7_1","volume-title":"Pvtv2: Improved Baselines with Pyramid Vision Transformer","author":"Wang W","year":"2022"},{"key":"e_1_2_8_8_1","doi-asserted-by":"crossref","unstructured":"LiuZ LinY CaoY et\u00a0al.Swin transformer: hierarchical vision transformer using shifted windows.Proceedings of the IEEE\/CVF International Conference on Computer Vision.2021.","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"e_1_2_8_9_1","first-page":"17864","article-title":"Per\u2010Pixel classification is not all you need for semantic segmentation","volume":"34","author":"Cheng B","year":"2021","journal-title":"Adv Neural Inf Process"},{"key":"e_1_2_8_10_1","first-page":"12077","article-title":"SegFormer: simple and efficient design for semantic segmentation with Transformers","volume":"34","author":"Xie E","year":"2021","journal-title":"Adv Neural Inf Process"},{"key":"e_1_2_8_11_1","doi-asserted-by":"crossref","unstructured":"LuoZL WangW XieE et\u00a0al.Panoptic SegFormer: delving deeper into panoptic segmentation with transformers.IEEE\/CVF Conference on Computer Vision and Pattern Recognition.2022.","DOI":"10.1109\/CVPR46437.2021.00542"},{"key":"e_1_2_8_12_1","doi-asserted-by":"crossref","unstructured":"WanZ ZhangJ ChenD LiaoJ.High\u2010fidelity pluralistic image completion with Transformers.IEEE\/CVF Conference on Computer Vision and Pattern Recognition.2022.","DOI":"10.1109\/ICCV48922.2021.00465"},{"key":"e_1_2_8_13_1","doi-asserted-by":"crossref","unstructured":"WangH ZhuY AdamH YuilleA ChenL\u2010C.MaX\u2010DeepLab: end\u2010to\u2010end panoptic segmentation with Mask Transformers.IEEE\/CVF Conference on Computer Vision and Pattern Recognition.2022.","DOI":"10.1109\/CVPR46437.2021.00542"},{"key":"e_1_2_8_14_1","doi-asserted-by":"crossref","unstructured":"HouB KaissisG SummersRM KainzB.Ratchet: medical transformer for chest x\u2010ray diagnosis and reporting.Medical Image Computing and Computer Assisted Intervention\u2013MICCAI 2021: 24th International Conference September 27\u2013October 1 2021 Proceedings Part VII 24 Strasbourg France.2021.Springer.","DOI":"10.1007\/978-3-030-87234-2_28"},{"key":"e_1_2_8_15_1","unstructured":"MatsoukasC HaslumJF S\u00f6derbergM SmithK Is it time to replace cnns with transformers for medical images?arXiv preprint arXiv:2108.09038.2021."},{"key":"e_1_2_8_16_1","first-page":"24617","article-title":"Federated split task\u2010agnostic vision transformer for COVID\u201019 CXR diagnosis","volume":"34","author":"Park S","year":"2021","journal-title":"Adv Neural Inf Process Sys"},{"key":"e_1_2_8_17_1","unstructured":"ChenJ LuY YuQ et\u00a0al.TransUNet: Transformers make strong encoders for medical image segmentation. arXiv.org.2021."},{"key":"e_1_2_8_18_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-031-08999-2_22"},{"key":"e_1_2_8_19_1","doi-asserted-by":"crossref","unstructured":"HatamizadehA TangY NathV et\u00a0al.UNETR: transformers for 3D medical image segmentation.IEEE\/CVF Winter Conference on Applications of Computer Vision (WACV).2022.","DOI":"10.1109\/WACV51458.2022.00181"},{"key":"e_1_2_8_20_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.media.2022.102615"},{"key":"e_1_2_8_21_1","unstructured":"ChenJ HeY FreyEC LiY DuY Vit\u2010v\u2010net: Vision transformer for unsupervised volumetric medical image registration. arXiv preprint arXiv:2104.06468 2021."},{"key":"e_1_2_8_22_1","first-page":"256","volume-title":"TransUNet+: Redesigning the skip connection to enhance features in medical image segmentation","author":"Liu Y","year":"2022"},{"key":"e_1_2_8_23_1","doi-asserted-by":"crossref","unstructured":"LinA ChenB XuJ ZhangZ LuG.DS\u2010TransUNet: Dual Swin Transformer U\u2010Net for Medical Image Segmentation. arXiv.org.2021.","DOI":"10.1109\/TIM.2022.3178991"},{"key":"e_1_2_8_24_1","unstructured":"ChangY MenghanH GuangtaoZ Xiao\u2010PingZ.TransClaw U\u2010Net: Claw U\u2010Net with Transformers for Medical Image Segmentation. ArXiv.2021. abs\/2107.05188."},{"key":"e_1_2_8_25_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-87583-1_7"},{"key":"e_1_2_8_26_1","doi-asserted-by":"crossref","unstructured":"XieY ZhangJ ShenC XiaY.CoTr: efficiently Bridging CNN and Transformer for 3D Medical Image Segmentation.International conference on medical image computing and computer\u2010assisted intervention.2021.","DOI":"10.1007\/978-3-030-87199-4_16"},{"key":"e_1_2_8_27_1","doi-asserted-by":"crossref","unstructured":"LiS SuiX LuoX XuX LiuY GohR.Medical image segmentation using Squeeze\u2010and\u2010Expansion Transformers. ArXiv.2021.abs\/2105.09511.","DOI":"10.24963\/ijcai.2021\/112"},{"key":"e_1_2_8_28_1","unstructured":"DosovitskiyA BeyerL KolesnikovA et\u00a0al.An image is worth 16\u00d716 words: transformers for image recognition at scale.arXiv 2020. arXiv preprint arXiv:2010.11929 2010."},{"key":"e_1_2_8_29_1","doi-asserted-by":"crossref","unstructured":"JiangJ TyagiN TringaleK CraneC VeeraraghavanH Self\u2010supervised 3D anatomy segmentation using self\u2010distilled masked image transformer (SMIT).Medical Image Computing and Computer Assisted Intervention\u2013MICCAI2022: 25th International Conference Singapore September 18\u201322 2022 Proceedings Part IV.2022.Springer.","DOI":"10.1007\/978-3-031-16440-8_53"},{"key":"e_1_2_8_30_1","doi-asserted-by":"crossref","unstructured":"TangY YangD LiW et\u00a0al.Self\u2010supervised pre\u2010training of swin transformers for 3d Med Image Anal.Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition.2022.","DOI":"10.1109\/CVPR52688.2022.02007"},{"key":"e_1_2_8_31_1","first-page":"12","volume-title":"MICCAI Multi\u2010Atlas Labeling Beyond Cranial Vault\u2014Workshop Challenge","author":"Landman B","year":"2015"},{"key":"e_1_2_8_32_1","doi-asserted-by":"publisher","DOI":"10.1038\/s41467-022-30695-9"},{"key":"e_1_2_8_33_1","doi-asserted-by":"crossref","unstructured":"ZhouL LiuH BaeJ HeJ SamarasD PrasannaP.Self pre\u2010training with masked autoencoders for Med Image Anal. arXiv preprint arXiv:2203.05573 2022.","DOI":"10.1109\/ISBI53787.2023.10230477"},{"key":"e_1_2_8_34_1","first-page":"63373","volume-title":"Deep Multimodal Representation Learning: A Survey","author":"Guo WZ","year":"2019"},{"key":"e_1_2_8_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/TRPMS.2018.2890359"},{"key":"e_1_2_8_36_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.sigpro.2015.01.001"},{"key":"e_1_2_8_37_1","doi-asserted-by":"crossref","unstructured":"WuS BondugulaS LuisierF ZhuangX NatarajanP.Zero\u2010shot event detection using multi\u2010modal fusion of weakly supervised concepts.Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2014.","DOI":"10.1109\/CVPR.2014.341"},{"key":"e_1_2_8_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2016.2627563"},{"key":"e_1_2_8_39_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neucom.2015.01.095"},{"key":"e_1_2_8_40_1","unstructured":"ZadehA ChenM PoriaS CambriaE MorencyL.Tensor fusion network for multimodal sentiment analysis. arXiv preprint arXiv:1707.07250.2017."},{"key":"e_1_2_8_41_1","volume-title":"Multimodal Deep Learning","author":"Ngiam J","year":"2011"},{"key":"e_1_2_8_42_1","unstructured":"WangW AroraR LivescuK BilmesJ.On Deep Multi\u2010View Representation Learning: Objectives and Optimization. ArXiv 2016. abs\/1602.01024."},{"key":"e_1_2_8_43_1","unstructured":"ZhuZ LuoP WangX TangX.Multi\u2010view perceptron: a deep model for learning face identity and view representations.28th Conference on Neural Information Processing Systems (NIPS).2014.Montreal Canada."},{"key":"e_1_2_8_44_1","doi-asserted-by":"crossref","unstructured":"CarneiroG NascimentoJ BradleyAP.Unregistered multiview mammogram analysis with pre\u2010trained deep learning models.18th International Conference on Medical Image Computing and Computer\u2010Assisted Intervention (MICCAI).2015.Munich Germany.","DOI":"10.1007\/978-3-319-24574-4_78"},{"key":"e_1_2_8_45_1","first-page":"115","volume-title":"Multimodal Deep Learning for Cervical Dysplasia Diagnosis","author":"Xu T","year":"2016"},{"key":"e_1_2_8_46_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.neuroimage.2014.06.077"},{"key":"e_1_2_8_47_1","doi-asserted-by":"crossref","unstructured":"KangY KimS ChoiS.Deep learning to hash with multiple representations.12th IEEE International Conference on Data Mining (ICDM).2012.Brussels Belgium.","DOI":"10.1109\/ICDM.2012.24"},{"key":"e_1_2_8_48_1","doi-asserted-by":"publisher","DOI":"10.1002\/mp.15610"},{"key":"e_1_2_8_49_1","doi-asserted-by":"crossref","unstructured":"GuoZ LiX HuangH GuoN LiQ Medical Image Segmentation Based On Multi\u2010Modal Convolutional Neural Network: Study On Image Fusion Schemes.15th IEEE International Symposium on Biomedical Imaging (ISBI).2018.Washington DC.","DOI":"10.1109\/ISBI.2018.8363717"},{"key":"e_1_2_8_50_1","doi-asserted-by":"publisher","DOI":"10.1088\/1361-6560\/ab440d"},{"key":"e_1_2_8_51_1","doi-asserted-by":"publisher","DOI":"10.1002\/mp.14903"},{"key":"e_1_2_8_52_1","doi-asserted-by":"crossref","unstructured":"CarneiroG NascimentoJ BradleyAP.Unregistered multiview mammogram analysis with pre\u2010trained deep learning models.International Conference on Medical Image Computing and Computer\u2010Assisted Intervention. Springer; 2015.","DOI":"10.1007\/978-3-319-24574-4_78"},{"key":"e_1_2_8_53_1","volume-title":"Overview of the HECKTOR Challenge at MICCAI 2021: Automatic Head and Neck Tumor Segmentation and Outcome Prediction in PET\/CT Images","author":"Andrearczyk V","year":"2022"},{"key":"e_1_2_8_54_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-98253-9_8"},{"key":"e_1_2_8_55_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPAMI.2017.2699184"},{"key":"e_1_2_8_56_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-98253-9_3"},{"key":"e_1_2_8_57_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-66182-7_79"},{"key":"e_1_2_8_58_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-98253-9_12"},{"key":"e_1_2_8_59_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-67194-5_4"},{"key":"e_1_2_8_60_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-98253-9_22"},{"key":"e_1_2_8_61_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-98253-9_7"},{"key":"e_1_2_8_62_1","doi-asserted-by":"crossref","unstructured":"ZhouH\u2010Y LuC YangS YuY.ConvNets vs. Transformers: whose visual representations are more transferable?Proceedings of the IEEE\/CVF International Conference on Computer Vision.2021.","DOI":"10.1109\/ICCVW54120.2021.00252"},{"key":"e_1_2_8_63_1","first-page":"23296","article-title":"Intriguing properties of vision transformers","volume":"34","author":"Naseer MM","year":"2021","journal-title":"Adv Neural Inf Process Sys"},{"key":"e_1_2_8_64_1","unstructured":"SimonyanK ZissermanA.Very deep convolutional networks for large\u2010scale image recognition. arXiv preprint arXiv:1409.1556 2014."},{"key":"e_1_2_8_65_1","unstructured":"YuF KoltunV.Multi\u2010scale context aggregation by dilated convolutions. arXiv preprint arXiv:1511.07122 2015."},{"key":"e_1_2_8_66_1","doi-asserted-by":"crossref","unstructured":"LiangM HuX.Recurrent convolutional neural network for object recognition.Proceedings of the IEEE conference on computer vision and pattern recognition.2015.","DOI":"10.1109\/CVPRW.2015.7301333"},{"key":"e_1_2_8_67_1","doi-asserted-by":"crossref","unstructured":"HeK ZhangX RenS SunJ.Deep residual learning for image recognition.Proceedings of the IEEE conference on computer vision and pattern recognition.2016.","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_8_68_1","first-page":"29","volume-title":"Understanding the effective receptive field in deep convolutional neural networks","author":"Luo W","year":"2016"},{"key":"e_1_2_8_69_1","unstructured":"VaswaniA ShazeerN PolosukhinI et\u00a0al.Attention Is All You Need.31st Annual Conference on Neural Information Processing Systems (NIPS).2017.Long Beach CA."},{"key":"e_1_2_8_70_1","unstructured":"DevlinJ ChangM LeeK ToutanovaK.Bert: pre\u2010training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 2018."},{"key":"e_1_2_8_71_1","first-page":"1877","article-title":"Language models are few\u2010shot learners","volume":"33","author":"Brown T","year":"2020","journal-title":"Adv Neural Inf Process Sys"},{"key":"e_1_2_8_72_1","doi-asserted-by":"crossref","unstructured":"WangX GirshickR GuptaA HeK.Non\u2010local neural networks.Proceedings of the IEEE conference on computer vision and pattern recognition.2018.","DOI":"10.1109\/CVPR.2018.00813"},{"key":"e_1_2_8_73_1","unstructured":"ParmarN VaswaniA UszkoreitJ et\u00a0al.Image transformer.International conference on machine learning.2018.PMLR."},{"key":"e_1_2_8_74_1","unstructured":"DosovitskiyA BeyerL KolesnikovA et\u00a0al.An image is worth 16\u00d716 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 2020."},{"key":"e_1_2_8_75_1","doi-asserted-by":"crossref","unstructured":"CarionN MassaF SynnaeveG UsunierN KirillovA ZagoruykoS.End\u2010to\u2010end object detection with transformers.European conference on computer vision.2020.Springer.","DOI":"10.1007\/978-3-030-58452-8_13"},{"key":"e_1_2_8_76_1","unstructured":"ChenM RadfordA SutskeverI et\u00a0al.Generative pretraining from pixels.Proceedings of the 37th International Conference on Machine Learning.HalDIII AartiS editors.2020 PMLR: Proceedings of Machine Learning Research. pp.1691\u20101703."},{"key":"e_1_2_8_77_1","doi-asserted-by":"crossref","unstructured":"ChenC\u2010FR FanQ PandaR.Crossvit: cross\u2010attention multi\u2010scale vision transformer for image classification.Proceedings of the IEEE\/CVF international conference on computer vision.2021.","DOI":"10.1109\/ICCV48922.2021.00041"},{"key":"e_1_2_8_78_1","doi-asserted-by":"crossref","unstructured":"TanH BansalM.Lxmert: learning cross\u2010modality encoder representations from transformers. arXiv preprint arXiv:1908.07490 2019.","DOI":"10.18653\/v1\/D19-1514"},{"key":"e_1_2_8_79_1","unstructured":"LiLH YatskarM YinD HsiehC\u2010J ChangK\u2010W.Visualbert: A simple and performant baseline for vision and language. arXiv preprint arXiv:1908.03557 2019."},{"key":"e_1_2_8_80_1","doi-asserted-by":"crossref","unstructured":"SunC MyersA VondrickC MurphyK SchmidC.Videobert: a joint model for video and language representation learning.Proceedings of the IEEE\/CVF International Conference on Computer Vision.2019.","DOI":"10.1109\/ICCV.2019.00756"},{"key":"e_1_2_8_81_1","doi-asserted-by":"crossref","unstructured":"ChenY\u2010C LiL YuL et\u00a0al.Uniter: universal image\u2010text representation learning.European conference on computer vision.2020.Springer.","DOI":"10.1007\/978-3-030-58577-8_7"},{"key":"e_1_2_8_82_1","doi-asserted-by":"crossref","unstructured":"LiX YinX LiC et\u00a0al.Oscar: object\u2010semantics aligned pre\u2010training for vision\u2010language tasks.European Conference on Computer Vision.2020.Springer.","DOI":"10.1007\/978-3-030-58577-8_8"},{"key":"e_1_2_8_83_1","doi-asserted-by":"crossref","unstructured":"PrakashA ChittaK GeigerA.Multi\u2010modal fusion transformer for end\u2010to\u2010end autonomous driving.Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition.2021.","DOI":"10.1109\/CVPR46437.2021.00700"},{"key":"e_1_2_8_84_1","doi-asserted-by":"crossref","unstructured":"HuangZ ZengZ HuangY LiuB FuD FuJ.Seeing out of the box: end\u2010to\u2010end pre\u2010training for vision\u2010language representation learning.Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition.2021.","DOI":"10.1109\/CVPR46437.2021.01278"},{"key":"e_1_2_8_85_1","doi-asserted-by":"crossref","unstructured":"HuR SinghA.Unit: multimodal multitask learning with a unified transformer.Proceedings of the IEEE\/CVF International Conference on Computer Vision.2021.","DOI":"10.1109\/ICCV48922.2021.00147"},{"key":"e_1_2_8_86_1","first-page":"24206","article-title":"Vatt: transformers for multimodal self\u2010supervised learning from raw video, audio and text","volume":"34","author":"Akbari H","year":"2021","journal-title":"Adv Neural Inf Process Sys"},{"key":"e_1_2_8_87_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00385"},{"key":"e_1_2_8_88_1","doi-asserted-by":"crossref","unstructured":"DingX ZhangX ZhouY HanJ DingG SunJ.Scaling up your kernels to 31\u00d731: revisiting large kernel design in cnns.Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition.2022.","DOI":"10.1109\/CVPR52688.2022.01166"},{"key":"e_1_2_8_89_1","doi-asserted-by":"crossref","unstructured":"LiY ChenJ JangS GongK LiQ.Investigation of Network Architecture for Multimodal Head\u2010and\u2010Neck Tumor Segmentation. Arxiv.org 2022.","DOI":"10.1109\/NSS\/MIC44845.2022.10399293"}],"container-title":["Medical Physics"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/aapm.onlinelibrary.wiley.com\/doi\/pdf\/10.1002\/mp.16703","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,3,12]],"date-time":"2024-03-12T11:07:39Z","timestamp":1710241659000},"score":1,"resource":{"primary":{"URL":"https:\/\/aapm.onlinelibrary.wiley.com\/doi\/10.1002\/mp.16703"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,9,30]]},"references-count":88,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,3]]}},"alternative-id":["10.1002\/mp.16703"],"URL":"https:\/\/doi.org\/10.1002\/mp.16703","archive":["Portico"],"relation":{},"ISSN":["0094-2405","2473-4209"],"issn-type":[{"value":"0094-2405","type":"print"},{"value":"2473-4209","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,9,30]]},"assertion":[{"value":"2023-02-22","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-07-30","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-09-30","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}