{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T02:01:26Z","timestamp":1760148086255,"version":"build-2065373602"},"reference-count":54,"publisher":"MDPI AG","issue":"7","license":[{"start":{"date-parts":[[2023,3,31]],"date-time":"2023-03-31T00:00:00Z","timestamp":1680220800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>This paper proposes a novel self-supervised based Cut-and-Paste GAN to perform foreground object segmentation and generate realistic composite images without manual annotations. We accomplish this goal by a simple yet effective self-supervised approach coupled with the U-Net discriminator. The proposed method extends the ability of the standard discriminators to learn not only the global data representations via classification (real\/fake) but also learn semantic and structural information through pseudo labels created using the self-supervised task. The proposed method empowers the generator to create meaningful masks by forcing it to learn informative per-pixel and global image feedback from the discriminator. Our experiments demonstrate that our proposed method significantly outperforms the state-of-the-art methods on the standard benchmark datasets.<\/jats:p>","DOI":"10.3390\/s23073649","type":"journal-article","created":{"date-parts":[[2023,3,31]],"date-time":"2023-03-31T08:27:27Z","timestamp":1680251247000},"page":"3649","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":2,"title":["SS-CPGAN: Self-Supervised Cut-and-Pasting Generative Adversarial Network for Object Segmentation"],"prefix":"10.3390","volume":"23","author":[{"given":"Kunal","family":"Chaturvedi","sequence":"first","affiliation":[{"name":"School of Computer Science, FEIT, University of Technology Sydney, Sydney, NSW 2007, Australia"}]},{"given":"Ali","family":"Braytee","sequence":"additional","affiliation":[{"name":"School of Computer Science, FEIT, University of Technology Sydney, Sydney, NSW 2007, Australia"}]},{"given":"Jun","family":"Li","sequence":"additional","affiliation":[{"name":"School of Computer Science, FEIT, University of Technology Sydney, Sydney, NSW 2007, Australia"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-7745-9667","authenticated-orcid":false,"given":"Mukesh","family":"Prasad","sequence":"additional","affiliation":[{"name":"School of Computer Science, FEIT, University of Technology Sydney, Sydney, NSW 2007, Australia"}]}],"member":"1968","published-online":{"date-parts":[[2023,3,31]]},"reference":[{"key":"ref_1","unstructured":"Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., and Bengio, Y. (2014, January 8\u201313). Generative adversarial nets. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, USA."},{"key":"ref_2","unstructured":"Brock, A., Donahue, J., and Simonyan, K. (2018). Large scale GAN training for high fidelity natural image synthesis. arXiv."},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Chaturvedi, K., Braytee, A., Vishwakarma, D.K., Saqib, M., Mery, D., and Prasad, M. (2021, January 18\u201322). Automated Threat Objects Detection with Synthetic Data for Real-Time X-ray Baggage Inspection. Proceedings of the 2021 International Joint Conference on Neural Networks (IJCNN), Shenzhen, China.","DOI":"10.1109\/IJCNN52387.2021.9533928"},{"key":"ref_4","unstructured":"Chen, M., Arti\u00e8res, T., and Denoyer, L. (2019, January 8\u201314). Unsupervised object segmentation by redrawing. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_5","unstructured":"Bielski, A., and Favaro, P. (2019, January 8\u201314). Emergence of object segmentation in perturbed generative models. Proceedings of the Advances in Neural Information Processing Systems, Vancouver, BC, Canada."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Abdal, R., Zhu, P., Mitra, N.J., and Wonka, P. (2021, January 10\u201317). Labels4free: Unsupervised segmentation using stylegan. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Montreal, BC, Canada.","DOI":"10.1109\/ICCV48922.2021.01371"},{"key":"ref_7","unstructured":"Arandjelovi\u0107, R., and Zisserman, A. (2019). Object discovery with a copy-pasting gan. arXiv."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"123","DOI":"10.1016\/j.isprsjprs.2022.02.008","article-title":"PolGAN: A deep-learning-based unsupervised forest height estimation based on the synergy of PolInSAR and LiDAR data","volume":"186","author":"Zhang","year":"2022","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"127844","DOI":"10.1016\/j.jhydrol.2022.127844","article-title":"An integrated inversion framework for heterogeneous aquifer structure identification with single-sample generative adversarial network","volume":"610","author":"Zhan","year":"2022","journal-title":"J. Hydrol."},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Zhou, G., Song, B., Liang, P., Xu, J., and Yue, T. (2022). Voids Filling of DEM with Multiattention Generative Adversarial Network Model. Remote Sens., 14.","DOI":"10.3390\/rs14051206"},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"738","DOI":"10.1016\/j.ins.2021.12.083","article-title":"SLC-GAN: An automated myocardial infarction detection model based on generative adversarial networks and convolutional neural networks with single-lead electrocardiogram synthesis","volume":"589","author":"Li","year":"2022","journal-title":"Inf. Sci."},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Fu, L., Li, J., Zhou, L., Ma, Z., Liu, S., Lin, Z., and Prasad, M. (2018, January 8\u201313). Utilizing Information from Task-Independent Aspects via GAN-Assisted Knowledge Transfer. Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil.","DOI":"10.1109\/IJCNN.2018.8489047"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Zhang, L., Li, J., Huang, T., Ma, Z., Lin, Z., and Prasad, M. (2018, January 8\u201313). GAN2C: Information Completion GAN with Dual Consistency Constraints. Proceedings of the 2018 International Joint Conference on Neural Networks (IJCNN), Rio de Janeiro, Brazil.","DOI":"10.1109\/IJCNN.2018.8489550"},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Chen, T., Zhai, X., Ritter, M., Lucic, M., and Houlsby, N. (2019, January 16\u201317). Self-supervised gans via auxiliary rotation loss. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01243"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Patel, P., Kumari, N., Singh, M., and Krishnamurthy, B. (2021, January 5\u20139). Lt-gan: Self-supervised gan with latent transformation detection. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, Virtual.","DOI":"10.1109\/WACV48630.2021.00323"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"Huang, R., Xu, W., Lee, T.Y., Cherian, A., Wang, Y., and Marks, T. (2020, January 1\u20135). Fx-gan: Self-supervised gan learning via feature exchange. Proceedings of the IEEE\/CVF Winter Conference on Applications of Computer Vision, Snowmass Village, CO, USA.","DOI":"10.1109\/WACV45572.2020.9093525"},{"key":"ref_17","unstructured":"Hou, L., Shen, H., Cao, Q., and Cheng, X. (2021, January 6\u201312). Self-Supervised GANs with Label Augmentation. Proceedings of the Advances in Neural Information Processing Systems, Online."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"4882","DOI":"10.1109\/TPAMI.2022.3186876","article-title":"Learning to Detect 3D Symmetry From Single-View RGB-D Images With Weak Supervision","volume":"45","author":"Shi","year":"2023","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"1398","DOI":"10.1111\/mice.12674","article-title":"Cross-scene pavement distress detection by a novel transfer learning framework","volume":"36","author":"Li","year":"2021","journal-title":"Comput.-Aided Civ. Infrastruct. Eng."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"3885","DOI":"10.1016\/j.apt.2021.08.038","article-title":"Efficient image segmentation based on deep learning for mineral image classification","volume":"32","author":"Liu","year":"2021","journal-title":"Adv. Powder Technol."},{"key":"ref_21","first-page":"173","article-title":"A Survey of Natural Language Generation","volume":"55","author":"Dong","year":"2022","journal-title":"ACM Comput. Surv."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-net: Convolutional networks for biomedical image segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"12633","DOI":"10.1109\/TITS.2021.3115823","article-title":"C2FDA: Coarse-to-Fine Domain Adaptation for Traffic Object Detection","volume":"23","author":"Zhang","year":"2022","journal-title":"IEEE Trans. Intell. Transp. Syst."},{"key":"ref_24","unstructured":"Yang, B., Gu, S., Zhang, B., Zhang, T., Chen, X., Sun, X., Chen, D., and Wen, F. (2022). Paint by Example: Exemplar-based Image Editing with Diffusion Models. arXiv."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Xie, B., Li, S., Lv, F., Liu, C.H., Wang, G., and Wu, D. (2022). A Collaborative Alignment Framework of Transferable Knowledge Extraction for Unsupervised Domain Adaptation. IEEE Trans. Knowl. Data Eng., Early Access.","DOI":"10.1109\/TKDE.2022.3185233"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Dang, W., Guo, J., Liu, M., Liu, S., Yang, B., Yin, L., and Zheng, W. (2022). A Semi-Supervised Extreme Learning Machine Algorithm Based on the New Weighted Kernel for Machine Smell. Appl. Sci., 12.","DOI":"10.3390\/app12189213"},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"42","DOI":"10.1109\/MSP.2021.3134634","article-title":"Self-Supervised Representation Learning: Introduction, advances, and challenges","volume":"39","author":"Ericsson","year":"2022","journal-title":"IEEE Signal Process. Mag."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"5536517","DOI":"10.1109\/TGRS.2022.3202908","article-title":"Self-Supervised Divide-and-Conquer Generative Adversarial Network for Classification of Hyperspectral Images","volume":"60","author":"Feng","year":"2022","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Baykal, G., and Unal, G. (2020, January 25\u201328). Deshufflegan: A self-supervised gan to improve structure learning. Proceedings of the 2020 IEEE International Conference on Image Processing (ICIP), Abu Dhabi, United Arab Emirates.","DOI":"10.1109\/ICIP40778.2020.9190774"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Thanh-Tung, H., and Tran, T. (2020, January 19\u201324). Catastrophic forgetting and mode collapse in GANs. Proceedings of the 2020 International Joint Conference on Neural Networks (IJCNN), Glasgow, UK.","DOI":"10.1109\/IJCNN48605.2020.9207181"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Mao, Q., Lee, H.Y., Tseng, H.Y., Ma, S., and Yang, M.H. (2019, January 15\u201320). Mode seeking generative adversarial networks for diverse image synthesis. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00152"},{"key":"ref_32","unstructured":"Tran, N.T., Tran, V.H., Nguyen, B.N., Yang, L., and Cheung, N.M.M. (2019, January 8\u201313). Self-supervised gan: Analysis and improvement with multi-class minimax game. Proceedings of the Advances in Neural Information Processing Systems, Montreal, QC, USA."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Xie, B., Li, S., Li, M., Liu, C., Huang, G., and Wang, G. (2023). SePiCo: Semantic-Guided Pixel Contrast for Domain Adaptive Semantic Segmentation. IEEE Trans. Pattern Anal. Mach. Intell., 1\u201317.","DOI":"10.1109\/TPAMI.2023.3237740"},{"key":"ref_34","first-page":"1443","article-title":"LFRSNet: A robust light field semantic segmentation network combining contextual and geometric features","volume":"10","author":"Yang","year":"2022","journal-title":"Front. Environ. Sci."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"7880","DOI":"10.1109\/TCSVT.2022.3187664","article-title":"UrbanLF: A Comprehensive Light Field Dataset for Semantic Segmentation of Urban Scenes","volume":"32","author":"Sheng","year":"2022","journal-title":"IEEE Trans. Circuits Syst. Video Technol."},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Chen, Y., Wei, Y., Wang, Q., Chen, F., Lu, C., and Lei, S. (2020). Mapping post-earthquake landslide susceptibility: A U-Net like approach. Remote Sens., 12.","DOI":"10.3390\/rs12172767"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Tran, L.A., and Le, M.H. (2019, January 20\u201321). Robust U-Net-based road lane markings detection for autonomous driving. Proceedings of the 2019 International Conference on System Science and Engineering (ICSSE), Dong Hoi, Vietnam.","DOI":"10.1109\/ICSSE.2019.8823532"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Huang, H., Lin, L., Tong, R., Hu, H., Zhang, Q., Iwamoto, Y., Han, X., Chen, Y.W., and Wu, J. (2020, January 4\u20138). UNet 3+: A Full-Scale Connected UNet for Medical Image Segmentation. Proceedings of the ICASSP 2020\u20142020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Barcelona, Spain.","DOI":"10.1109\/ICASSP40776.2020.9053405"},{"key":"ref_39","unstructured":"Welinder, P., Branson, S., Mita, T., Wah, C., Schroff, F., Belongie, S., and Perona, P. (2020, November 04). Caltech-UCSD Birds 200. Available online: https:\/\/www.vision.caltech.edu\/datasets\/cub_200_2011\/."},{"key":"ref_40","unstructured":"Nilsback, M.E., and Zisserman, A. (2020, November 04). Automated Flower Classification over a Large Number of Classes. Available online: https:\/\/www.robots.ox.ac.uk\/~vgg\/data\/flowers\/102\/."},{"key":"ref_41","unstructured":"Maji, S., Rahtu, E., Kannala, J., Blaschko, M., and Vedaldi, A. (2020, November 04). Fine-Grained Visual Classification of Aircraft. Available online: https:\/\/www.robots.ox.ac.uk\/~vgg\/data\/fgvc-aircraft\/."},{"key":"ref_42","unstructured":"Zhou, B., Lapedriza, A., Khosla, A., Oliva, A., and Torralba, A. (2020, November 04). Places: A 10 Million Image Database for Scene Recognition. Available online: http:\/\/places2.csail.mit.edu\/download.html."},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Dev, S., Lee, Y.H., and Winkler, S. (2020, November 04). Categorization of Cloud Image Patches Using an Improved Texton-Based Approach. Available online: https:\/\/stefan.winkler.site\/Publications\/icip2015cat.pdf.","DOI":"10.1109\/ICIP.2015.7350833"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"309","DOI":"10.1145\/1015706.1015720","article-title":"\u201cGrabCut\u201d interactive foreground extraction using iterated graph cuts","volume":"23","author":"Rother","year":"2004","journal-title":"ACM Trans. Graph. (TOG)"},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"1947","DOI":"10.1109\/TPAMI.2018.2856256","article-title":"Stackgan++: Realistic image synthesis with stacked generative adversarial networks","volume":"41","author":"Zhang","year":"2018","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Benny, Y., and Wolf, L. (2020, January 23\u201328). Onegan: Simultaneous unsupervised learning of conditional image generation, foreground segmentation, and fine-grained clustering. Proceedings of the European Conference on Computer Vision, Glasgow, UK.","DOI":"10.1007\/978-3-030-58574-7_31"},{"key":"ref_47","unstructured":"Yang, J., Kannan, A., Batra, D., and Parikh, D. (2017). Lr-gan: Layered recursive generative adversarial networks for image generation. arXiv."},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Yang, Y., Bilen, H., Zou, Q., Cheung, W.Y., and Ji, X. (2021). Unsupervised Foreground-Background Segmentation with Equivariant Layered GANs. arXiv.","DOI":"10.1109\/WACV51458.2022.00044"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Singh, K.K., Ojha, U., and Lee, Y.J. (2019, January 15\u201319). Finegan: Unsupervised hierarchical disentanglement for fine-grained object generation and discovery. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00665"},{"key":"ref_50","unstructured":"Mo, S., Kang, H., Sohn, K., Li, C.L., and Shin, J. (2021, January 6\u201312). Object-aware contrastive learning for debiased scene representation. Proceedings of the Advances in Neural Information Processing Systems, Online."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"8055","DOI":"10.1109\/TIP.2020.3011269","article-title":"Unsupervised learning of image segmentation based on differentiable feature clustering","volume":"29","author":"Kim","year":"2020","journal-title":"IEEE Trans. Image Process."},{"key":"ref_52","unstructured":"Ji, X., Henriques, J.F., and Vedaldi, A. (November, January 27). Invariant information clustering for unsupervised image classification and segmentation. Proceedings of the IEEE\/CVF International Conference on Computer Vision, Seoul, Republic of Korea."},{"key":"ref_53","unstructured":"Melas-Kyriazi, L., Rupprecht, C., Laina, I., and Vedaldi, A. (2021). Finding an unsupervised image segmenter in each of your deep generative models. arXiv."},{"key":"ref_54","unstructured":"Voynov, A., Morozov, S., and Babenko, A. (2021, January 18\u201324). Object segmentation without labels with large-scale generative models. Proceedings of the International Conference on Machine Learning, PMLR, Virtual."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/7\/3649\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T19:07:37Z","timestamp":1760123257000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/23\/7\/3649"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,3,31]]},"references-count":54,"journal-issue":{"issue":"7","published-online":{"date-parts":[[2023,4]]}},"alternative-id":["s23073649"],"URL":"https:\/\/doi.org\/10.3390\/s23073649","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2023,3,31]]}}}