{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T04:59:47Z","timestamp":1750309187673,"version":"3.41.0"},"reference-count":35,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2024,4,22]],"date-time":"2024-04-22T00:00:00Z","timestamp":1713744000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"name":"National Science and Technology Council","award":["111-2636-E-006-026-, 112-2221-E-006-100- and 112-2221-E-006-150-MY3"],"award-info":[{"award-number":["111-2636-E-006-026-, 112-2221-E-006-100- and 112-2221-E-006-150-MY3"]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Comput. Healthcare"],"published-print":{"date-parts":[[2024,4,30]]},"abstract":"<jats:p>This article explores the application of self-supervised contrastive learning in the medical domain, focusing on classification of multi-modality Magnetic Resonance (MR) images. To address the challenges of limited and hard-to-annotate medical data, we introduce multi-modality data augmentation (MDA) and cross-modality group convolution (CGC). In the pre-training phase, we leverage Simple Siamese networks to maximize the similarity between two augmented MR images from a patient, without a handcrafted pretext task. Our approach also combines 3D and 2D group convolution with a channel shuffle operation to efficiently incorporate different modalities of image features. Evaluation on liver MR images from a well-known hospital in Taiwan demonstrates a significant improvement over previous methods. This work contributes to advancing multi-modality contrastive learning, particularly in the context of medical imaging, offering enhanced tools for analyzing complex image data.<\/jats:p>","DOI":"10.1145\/3639414","type":"journal-article","created":{"date-parts":[[2023,12,30]],"date-time":"2023-12-30T16:09:07Z","timestamp":1703952547000},"page":"1-13","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["Enhancing Robust Liver Cancer Diagnosis: A Contrastive Multi-Modality Learner with Lightweight Fusion and Effective Data Augmentation"],"prefix":"10.1145","volume":"5","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-6014-4191","authenticated-orcid":false,"given":"Pei-Xuan","family":"Li","sequence":"first","affiliation":[{"name":"Department of Electrical Engineering, National Cheng Kung University, Taiwan, Taiwan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-6924-1337","authenticated-orcid":false,"given":"Hsun-Ping","family":"Hsieh","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, National Cheng Kung University, Taiwan, Taiwan"}]},{"ORCID":"https:\/\/orcid.org\/0009-0002-7823-3755","authenticated-orcid":false,"given":"Yang","family":"Fan-Chiang","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, National Cheng Kung University, Taiwan, Taiwan"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-7499-0460","authenticated-orcid":false,"given":"Ding-You","family":"Wu","sequence":"additional","affiliation":[{"name":"Department of Electrical Engineering, National Cheng Kung University, Taiwan, Taiwan"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0721-9606","authenticated-orcid":false,"given":"Ching-Chung","family":"Ko","sequence":"additional","affiliation":[{"name":"Department of Medical Imaging, Chi Mei Medical Center, Taiwan, Tainan, Taiwan and Department of Health and Nutrition, Chia Nan University of Pharmacy and Science, Taiwan, Tainan, Taiwan and Institute of Biomedical Sciences, National Sun Yat-Sen University, Taiwan, Kaohsiung, Taiwan"}]}],"member":"320","published-online":{"date-parts":[[2024,4,22]]},"reference":[{"key":"e_1_3_2_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV48922.2021.00346"},{"key":"e_1_3_2_3_2","first-page":"1597","volume-title":"International Conference on Machine Learning","author":"Chen Ting","year":"2020","unstructured":"Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In International Conference on Machine Learning. PMLR, 1597\u20131607."},{"key":"e_1_3_2_4_2","article-title":"Big self-supervised models are strong semi-supervised learners","author":"Chen Ting","year":"2020","unstructured":"Ting Chen, Simon Kornblith, Kevin Swersky, Mohammad Norouzi, and Geoffrey E. Hinton. 2020. Big self-supervised models are strong semi-supervised learners. In Advances in Neural Information Processing Systems, Vol. 33, 22243\u201322255.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_5_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR46437.2021.01549"},{"key":"e_1_3_2_6_2","article-title":"Bert: Pre-training of deep bidirectional transformers for language understanding","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of naacL-HLT, Vol. 1. 2.","journal-title":"Proceedings of naacL-HLT"},{"key":"e_1_3_2_7_2","first-page":"285","volume-title":"International Conference on Medical Image Computing and Computer-Assisted Intervention","author":"Fidon Lucas","year":"2017","unstructured":"Lucas Fidon, Wenqi Li, Luis C. Garcia-Peraza-Herrera, Jinendra Ekanayake, Neil Kitchen, Sebastien Ourselin, and Tom Vercauteren. 2017. Scalable multimodal convolutional networks for brain tumour segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 285\u2013293."},{"key":"e_1_3_2_8_2","article-title":"Generative adversarial nets","author":"Goodfellow Ian","year":"2014","unstructured":"Ian Goodfellow, Jean Pouget Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in Neural Information Processing Systems, Vol. 27, Curran Associates, Inc., 2672\u20132680.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_9_2","article-title":"Bootstrap your own latent-a new approach to self-supervised learning","author":"Grill Jean-Bastien","year":"2020","unstructured":"Jean-Bastien Grill, Florian Strub, Florent Altch\u00e9, Corentin Tallec, Pierre Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, R\u00e9mi Munos, and Michal Valko. 2020. Bootstrap your own latent-a new approach to self-supervised learning. In Advances in Neural Information Processing Systems, Vol. 33, 21271\u201321284.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_10_2","first-page":"903","volume-title":"2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI \u201918)","author":"Guo Zhe","year":"2018","unstructured":"Zhe Guo, Xiang Li, Heng Huang, Ning Guo, and Quanzheng Li. 2018. Medical image segmentation based on multi-modal convolutional neural network: Study on image fusion schemes. In 2018 IEEE 15th International Symposium on Biomedical Imaging (ISBI \u201918). IEEE, 903\u2013907."},{"key":"e_1_3_2_11_2","doi-asserted-by":"publisher","DOI":"10.1186\/s40537-019-0263-7"},{"key":"e_1_3_2_12_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR42600.2020.00975"},{"key":"e_1_3_2_13_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_2_14_2","first-page":"448","volume-title":"International Conference on Machine Learning","author":"Ioffe Sergey","year":"2015","unstructured":"Sergey Ioffe and Christian Szegedy. 2015. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International Conference on Machine Learning. PMLR, 448\u2013456."},{"key":"e_1_3_2_15_2","first-page":"777","volume-title":"International Conference on Medical Image Computing and Computer-Assisted Intervention","author":"Jiang Jue","year":"2018","unstructured":"Jue Jiang, Yu-Chi Hu, Neelam Tyagi, Pengpeng Zhang, Andreas Rimner, Gig S. Mageras, Joseph O Deasy, and Harini Veeraraghavan. 2018. Tumor-aware, adversarial domain adaptation from ct to mri for lung cancer segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 777\u2013785."},{"issue":"30","key":"e_1_3_2_16_2","article-title":"LayerCAM: Exploring hierarchical class activation maps for localization","volume":"30","author":"Jiang Peng-Tao","year":"2021","unstructured":"Peng-Tao Jiang, Chang-Bin Zhang, Qibin Hou, Ming-Ming Cheng, and Yunchao Wei. 2021. LayerCAM: Exploring hierarchical class activation maps for localization. IEEE Transactions on Image Processing 30, 30 (2021), 5875\u20135888.","journal-title":"IEEE Transactions on Image Processing"},{"key":"e_1_3_2_17_2","doi-asserted-by":"crossref","unstructured":"Konstantinos Kamnitsas Christian Ledig Virginia F. J. Newcombe Joanna P. Simpson Andrew D. Kane David K. Menon Daniel Rueckert and Ben Glocker. 2017. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation. Medical Image Analysis 36 22 (2017) 61\u201378.","DOI":"10.1016\/j.media.2016.10.004"},{"key":"e_1_3_2_18_2","first-page":"116","volume-title":"Proceedings of the Conference on Health, Inference, and Learning","author":"Ke Alexander","year":"2021","unstructured":"Alexander Ke, William Ellsworth, Oishi Banerjee, Andrew Y. Ng, and Pranav Rajpurkar. 2021. CheXtransfer: Performance and parameter efficiency of ImageNet models for chest X-Ray interpretation. In Proceedings of the Conference on Health, Inference, and Learning. 116\u2013124."},{"key":"e_1_3_2_19_2","volume-title":"International Conference on Learning Representations (ICLR\u201918)","author":"Komodakis Nikos","year":"2018","unstructured":"Nikos Komodakis and Spyros Gidaris. 2018. Unsupervised representation learning by predicting image rotations. In International Conference on Learning Representations (ICLR\u201918)."},{"key":"e_1_3_2_20_2","article-title":"ImageNet classification with deep convolutional neural networks","author":"Krizhevsky Alex","year":"2012","unstructured":"Alex Krizhevsky, Ilya Sutskever, and Geoffrey E. Hinton. 2012. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems, F. Pereira, C. J. Burges, L. Bottou, and K. Q. Weinberger (Eds.). Vol. 25, Curran Associates, Inc., 1097\u20131105.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_21_2","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i01.5421"},{"key":"e_1_3_2_22_2","volume-title":"2nd International Conference on Learning Representations (ICLR\u201914, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings)","author":"Lin Min","year":"2014","unstructured":"Min Lin, Qiang Chen, and Shuicheng Yan. 2014. Network in network. In 2nd International Conference on Learning Representations (ICLR\u201914, Banff, AB, Canada, April 14-16, 2014, Conference Track Proceedings), Yoshua Bengio and Yann LeCun (Eds.)."},{"issue":"22","key":"e_1_3_2_23_2","article-title":"A survey on deep learning in medical image analysis","volume":"42","author":"Litjens Geert","year":"2017","unstructured":"Geert Litjens, Thijs Kooi, Babak Ehteshami Bejnordi, Arnaud Arindra Adiyoso Setio, Francesco Ciompi, Mohsen Ghafoorian, Jeroen Awm Van Der Laak, Bram Van Ginneken, and Clara I. S\u00e1nchez. 2017. A survey on deep learning in medical image analysis. Medical Image Analysis 42, 22 (2017), 60\u201388.","journal-title":"Medical Image Analysis"},{"key":"e_1_3_2_24_2","unstructured":"Prajit Ramachandran Barret Zoph and Quoc V. Le. 2017. Searching for activation functions. arXiv:1710.05941. Retrieved from https:\/\/arxiv.org\/abs\/cs\/1710.05941"},{"key":"e_1_3_2_25_2","unstructured":"Hari Sowrirajan Jingbo Yang Andrew Y. Ng and Pranav Rajpurkar. 2021. MoCo pretraining improves representation and transferability of chest x-ray models. In Proceedings of the 4th Conference on Medical Imaging with Deep Learning (Proceedings of Machine Learning Research Vol. 143) Mattias Heinrich Qi Dou Marleen de Bruijne Jan Lellmann Alexander Schl\u00e4fer and Floris Ernst (Eds.). PMLR 728\u2013744."},{"key":"e_1_3_2_26_2","article-title":"CAiD: Context-Aware instance discrimination for self-supervised learning in medical imaging","author":"Taher Mohammad Reza Hosseinzadeh","year":"2022","unstructured":"Mohammad Reza Hosseinzadeh Taher, Fatemeh Haghighi, Michael B. Gotway, and Jianming Liang. 2022. CAiD: Context-Aware instance discrimination for self-supervised learning in medical imaging. In Proceedings of Machine Learning Research 172 (2022), 535\u2013551. Publisher Copyright: 2022 M. R. Hosseinzadeh Taher, F. Haghighi, M. B. Gotway J. Liang.; 5th International Conference on Medical Imaging with Deep Learning, MIDL 2022; Conference date: 06-07-2022 Through 08-07-2022.","journal-title":"Proceedings of Machine Learning Research"},{"key":"e_1_3_2_27_2","unstructured":"Mohammad Reza Hosseinzadeh Taher Fatemeh Haghighi Michael B. Gotway and Jianming Liang. 2022. CAiD: Context-Aware Instance Discrimination for Self-supervised Learning in Medical Imaging. arXiv:2204.07344. Retrieved from https:\/\/arxiv.org\/abs\/cs\/2204.07344"},{"key":"e_1_3_2_28_2","doi-asserted-by":"crossref","first-page":"661","DOI":"10.1007\/978-3-030-78191-0_51","volume-title":"International Conference on Information Processing in Medical Imaging","author":"Taleb Aiham","year":"2021","unstructured":"Aiham Taleb, Christoph Lippert, Tassilo Klein, and Moin Nabi. 2021. Multimodal self-supervised learning for medical image analysis. In International Conference on Information Processing in Medical Imaging. Springer, 661\u2013673."},{"key":"e_1_3_2_29_2","first-page":"6105","volume-title":"International Conference on Machine Learning","author":"Tan Mingxing","year":"2019","unstructured":"Mingxing Tan and Quoc Le. 2019. Efficientnet: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning. PMLR, 6105\u20136114."},{"key":"e_1_3_2_30_2","first-page":"13806","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition","author":"Tschannen Michael","year":"2020","unstructured":"Michael Tschannen, Josip Djolonga, Marvin Ritter, Aravindh Mahendran, Neil Houlsby, Sylvain Gelly, and Mario Lucic. 2020. Self-supervised learning of video-induced visual invariances. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 13806\u201313815."},{"key":"e_1_3_2_31_2","first-page":"6393","volume-title":"Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","author":"Tseng Kuan-Lun","year":"2017","unstructured":"Kuan-Lun Tseng, Yen-Liang Lin, Winston Hsu, and Chung-Yang Huang. 2017. Joint sequence learning and cross-modality convolution for 3d biomedical segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6393\u20136400."},{"key":"e_1_3_2_32_2","unstructured":"Yen Nhi Truong Vu Richard Wang Niranjan Balachandar Can Liu Andrew Y. Ng and Pranav Rajpurkar. 2021. Medaug: Contrastive learning leveraging patient metadata improves representations for chest x-ray interpretation. In Machine Learning for Healthcare Conference PMLR 755\u2013769."},{"key":"e_1_3_2_33_2","first-page":"9929","volume-title":"International Conference on Machine Learning","author":"Wang Tongzhou","year":"2020","unstructured":"Tongzhou Wang and Phillip Isola. 2020. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In International Conference on Machine Learning. PMLR, 9929\u20139939."},{"key":"e_1_3_2_34_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-319-46487-9_40"},{"key":"e_1_3_2_35_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00716"},{"key":"e_1_3_2_36_2","volume-title":"Proceedings of the IEEE\/CVF International Conference on Computer Vision","author":"Zhou Hong-Yu","year":"2021","unstructured":"Hong-Yu Zhou, Chixiang Lu, Sibei Yang, Xiaoguang Han, and Yizhou Yu. 2021. Preservational learning improves self-supervised medical image models by reconstructing diverse contexts. In Proceedings of the IEEE\/CVF International Conference on Computer Vision, 3499\u20133509."}],"container-title":["ACM Transactions on Computing for Healthcare"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3639414","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3639414","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T22:54:11Z","timestamp":1750287251000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3639414"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4,22]]},"references-count":35,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2024,4,30]]}},"alternative-id":["10.1145\/3639414"],"URL":"https:\/\/doi.org\/10.1145\/3639414","relation":{},"ISSN":["2691-1957","2637-8051"],"issn-type":[{"type":"print","value":"2691-1957"},{"type":"electronic","value":"2637-8051"}],"subject":[],"published":{"date-parts":[[2024,4,22]]},"assertion":[{"value":"2022-10-27","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-12-10","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2024-04-22","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}