{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,5,1]],"date-time":"2026-05-01T19:32:02Z","timestamp":1777663922319,"version":"3.51.4"},"reference-count":38,"publisher":"MDPI AG","issue":"21","license":[{"start":{"date-parts":[[2024,10,28]],"date-time":"2024-10-28T00:00:00Z","timestamp":1730073600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["42171304"],"award-info":[{"award-number":["42171304"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["SKL_SGIIT_20240303"],"award-info":[{"award-number":["SKL_SGIIT_20240303"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["KLSMNR-K202301"],"award-info":[{"award-number":["KLSMNR-K202301"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["D040404"],"award-info":[{"award-number":["D040404"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]},{"name":"Open Research Fund of the State Key Laboratory of Space\u2013Earth Integrated Information Technology","award":["42171304"],"award-info":[{"award-number":["42171304"]}]},{"name":"Open Research Fund of the State Key Laboratory of Space\u2013Earth Integrated Information Technology","award":["SKL_SGIIT_20240303"],"award-info":[{"award-number":["SKL_SGIIT_20240303"]}]},{"name":"Open Research Fund of the State Key Laboratory of Space\u2013Earth Integrated Information Technology","award":["KLSMNR-K202301"],"award-info":[{"award-number":["KLSMNR-K202301"]}]},{"name":"Open Research Fund of the State Key Laboratory of Space\u2013Earth Integrated Information Technology","award":["D040404"],"award-info":[{"award-number":["D040404"]}]},{"name":"Key Laboratory of Land Satellite Remote Sensing Application, Ministry of Natural Resources of the People\u2019s Republic of China","award":["42171304"],"award-info":[{"award-number":["42171304"]}]},{"name":"Key Laboratory of Land Satellite Remote Sensing Application, Ministry of Natural Resources of the People\u2019s Republic of China","award":["SKL_SGIIT_20240303"],"award-info":[{"award-number":["SKL_SGIIT_20240303"]}]},{"name":"Key Laboratory of Land Satellite Remote Sensing Application, Ministry of Natural Resources of the People\u2019s Republic of China","award":["KLSMNR-K202301"],"award-info":[{"award-number":["KLSMNR-K202301"]}]},{"name":"Key Laboratory of Land Satellite Remote Sensing Application, Ministry of Natural Resources of the People\u2019s Republic of China","award":["D040404"],"award-info":[{"award-number":["D040404"]}]},{"name":"Civil Space Advance Research Project of China","award":["42171304"],"award-info":[{"award-number":["42171304"]}]},{"name":"Civil Space Advance Research Project of China","award":["SKL_SGIIT_20240303"],"award-info":[{"award-number":["SKL_SGIIT_20240303"]}]},{"name":"Civil Space Advance Research Project of China","award":["KLSMNR-K202301"],"award-info":[{"award-number":["KLSMNR-K202301"]}]},{"name":"Civil Space Advance Research Project of China","award":["D040404"],"award-info":[{"award-number":["D040404"]}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Remote Sensing"],"abstract":"<jats:p>Satellite remote sensing images contain complex and diverse ground object information and the images exhibit spatial multi-scale characteristics, making the panoptic segmentation of satellite remote sensing images a highly challenging task. Due to the lack of large-scale annotated datasets for panoramic segmentation, existing methods still suffer from weak model generalization capabilities. To mitigate this issue, this paper leverages the advantages of the Segment Anything Model (SAM), which can segment any object in remote sensing images without requiring any annotations and proposes a high-resolution remote sensing image panoptic segmentation method called Remote Sensing Panoptic Segmentation SAM (RSPS-SAM). Firstly, to address the problem of global information loss caused by cropping large remote sensing images for training, a Batch Attention Pyramid was designed to extract multi-scale features from remote sensing images and capture long-range contextual information between cropped patches, thereby enhancing the semantic understanding of remote sensing images. Secondly, we constructed a Mask Decoder to address the limitation of SAM requiring manual input prompts and its inability to output category information. This decoder utilized mask-based attention for mask segmentation, enabling automatic prompt generation and category prediction of segmented objects. Finally, the effectiveness of the proposed method was validated on the high-resolution remote sensing image airport scene dataset RSAPS-ASD. The results demonstrate that the proposed method achieves segmentation and recognition of foreground instances and background regions in high-resolution remote sensing images without the need for prompt input, while providing smooth segmentation boundaries with a panoptic segmentation quality (PQ) of 57.2, outperforming current mainstream methods.<\/jats:p>","DOI":"10.3390\/rs16214002","type":"journal-article","created":{"date-parts":[[2024,10,28]],"date-time":"2024-10-28T07:47:18Z","timestamp":1730101638000},"page":"4002","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":15,"title":["RSPS-SAM: A Remote Sensing Image Panoptic Segmentation Method Based on SAM"],"prefix":"10.3390","volume":"16","author":[{"given":"Zhuoran","family":"Liu","sequence":"first","affiliation":[{"name":"School of Geodesy and Geomatics, Beijing University of Civil Engineering and Architecture, Beijing 102627, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zizhen","family":"Li","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Space-Earth Integrated Information Technology, Beijing Institute of Satellite Information Engineering, Beijing 100095, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ying","family":"Liang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Space-Earth Integrated Information Technology, Beijing Institute of Satellite Information Engineering, Beijing 100095, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3742-5398","authenticated-orcid":false,"given":"Claudio","family":"Persello","sequence":"additional","affiliation":[{"name":"Department of Earth Observation Science, Faculty ITC, University of Twente, 7500AE Enschede, The Netherlands"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Bo","family":"Sun","sequence":"additional","affiliation":[{"name":"College of Information and Communication Engineering, Harbin Engineering University, Harbin 150001, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Guangjun","family":"He","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Space-Earth Integrated Information Technology, Beijing Institute of Satellite Information Engineering, Beijing 100095, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8331-7200","authenticated-orcid":false,"given":"Lei","family":"Ma","sequence":"additional","affiliation":[{"name":"School of Geography and Ocean Science, Nanjing University, Nanjing 210023, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,10,28]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"1043","DOI":"10.1007\/s11430-012-4445-9","article-title":"Current Issues in High-Resolution Earth Observation Technology","volume":"55","author":"Li","year":"2012","journal-title":"Sci. China Earth Sci."},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Kirillov, A., He, K., Girshick, R., Rother, C., and Dollar, P. (2019, January 16\u201320). Panoptic Segmentation. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00963"},{"key":"ref_3","doi-asserted-by":"crossref","unstructured":"Fare Garnot, V.S., and Landrieu, L. (2021, January 11\u201317). Panoptic Segmentation of Satellite Image Time Series with Convolutional Temporal Attention Networks. Proceedings of the 2021 IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.00483"},{"key":"ref_4","first-page":"66","article-title":"Autonomous Remote Sensing Investigation and Monitoring Technique of Typical Classes of Natural Resources and Its Application","volume":"29","author":"Zhang","year":"2022","journal-title":"Geomat. World"},{"key":"ref_5","first-page":"33","article-title":"Parameter selection experiment of urban block object segmentation based on Landsat 8","volume":"30","author":"Xu","year":"2023","journal-title":"J. Spatio Temporal Inf."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"1660","DOI":"10.1109\/LRA.2023.3346760","article-title":"Panoptic Segmentation with Partial Annotations for Agricultural Robots","volume":"9","author":"Weyler","year":"2024","journal-title":"IEEE Robot. Autom. Lett."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"103283","DOI":"10.1016\/j.dsp.2021.103283","article-title":"A Survey on Deep Learning-Based Panoptic Segmentation","volume":"120","author":"Li","year":"2022","journal-title":"Digit. Signal Process."},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Sakaino, H. (2023, January 20\u201322). PanopticRoad: Integrated Panoptic Road Segmentation Under Adversarial Conditions. Proceedings of the 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Vancouver, BC, Canada.","DOI":"10.1109\/CVPRW59228.2023.00367"},{"key":"ref_9","doi-asserted-by":"crossref","unstructured":"De Carvalho, O.L.F., De Carvalho J\u00fanior, O.A., Silva, C.R.E., De Albuquerque, A.O., Santana, N.C., Borges, D.L., Gomes, R.A.T., and Guimar\u00e3es, R.F. (2022). Panoptic Segmentation Meets Remote Sensing. Remote Sens., 14.","DOI":"10.3390\/rs14040965"},{"key":"ref_10","first-page":"1","article-title":"Panoptic Perception: A Novel Task and Fine-Grained Dataset for Universal Remote Sensing Image Interpretation","volume":"62","author":"Zhao","year":"2024","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"107515","DOI":"10.1016\/j.asoc.2021.107515","article-title":"Cascaded Panoptic Segmentation Method for High Resolution Remote Sensing Image","volume":"109","author":"Hua","year":"2021","journal-title":"Appl. Soft Comput."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"7798","DOI":"10.1080\/01431161.2021.1966853","article-title":"Building Panoptic Change Segmentation with the Use of Uncertainty Estimation in Squeeze-and-Attention CNN and Remote Sensing Observations","volume":"42","year":"2021","journal-title":"Int. J. Remote Sens."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TGRS.2023.3268606","article-title":"Towards On-Board Panoptic Segmentation of Multispectral Satellite Images","volume":"61","author":"Fernando","year":"2023","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"\u0160ari\u0107, J., Or\u0161i\u0107, M., and \u0160egvi\u0107, S. (2023). Panoptic SwiftNet: Pyramidal Fusion for Real-Time Panoptic Segmentation. Remote Sens., 15.","DOI":"10.3390\/rs15081968"},{"key":"ref_15","unstructured":"Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., and Clark, J. (2021, January 18\u201324). Learning Transferable Visual Models from Natural Language Supervision. Proceedings of the 38th International Conference on Machine Learning, Online."},{"key":"ref_16","unstructured":"Yuan, L., Chen, D., Chen, Y.-L., Codella, N., Dai, X., Gao, J., Hu, H., Huang, X., Li, B., and Li, C. (2021). Florence: A New Foundation Model for Computer Vision. arXiv."},{"key":"ref_17","unstructured":"Bao, H., Dong, L., Piao, S., and Wei, F. (2021). BEiT: BERT Pre-Training of Image Transformers. arXiv."},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., and Lo, W.-Y. (2023, January 2\u20136). Segment Anything. Proceedings of the 2023 IEEE\/CVF International Conference on Computer Vision (ICCV), Paris, France.","DOI":"10.1109\/ICCV51070.2023.00371"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ren, Y., Yang, X., Wang, Z., Yu, G., Liu, Y., Liu, X., Meng, D., Zhang, Q., and Yu, G. (2023). Segment Anything Model (SAM) Assisted Remote Sensing Supervision for Mariculture\u2014Using Liaoning Province, China as an Example. Remote Sens., 15.","DOI":"10.3390\/rs15245781"},{"key":"ref_20","unstructured":"Wu, J., Ji, W., Liu, Y., Fu, H., Xu, M., Xu, Y., and Jin, Y. (2023). Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation. arXiv."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Zhao, Z. (2023, January 27\u201329). Enhancing Autonomous Driving with Grounded-Segment Anything Model: Limitations and Mitigations. Proceedings of the 2023 IEEE 3rd International Conference on Data Science and Computer Application (ICDSCA), Dalian, China.","DOI":"10.1109\/ICDSCA59871.2023.10393594"},{"key":"ref_22","first-page":"8815","article-title":"SAMRS: Scaling-up Remote Sensing Segmentation Dataset with Segment Anything Model","volume":"Volume 36","author":"Oh","year":"2023","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"ref_23","first-page":"1","article-title":"Make Segment Anything Model Perfect on Shadow Detection","volume":"61","author":"Chen","year":"2023","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Qian, X., Lin, C., Chen, Z., and Wang, W. (2024). SAM-Induced Pseudo Fully Supervised Learning for Weakly Supervised Object Detection in Remote Sensing Images. Remote Sens., 16.","DOI":"10.3390\/rs16091532"},{"key":"ref_25","first-page":"103540","article-title":"The Segment Anything Model (SAM) for Remote Sensing Applications: From Zero to One Shot","volume":"124","author":"Osco","year":"2023","journal-title":"Int. J. Appl. Earth Obs. Geoinf."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"617","DOI":"10.1007\/s11633-023-1385-0","article-title":"Segment Anything Is Not Always Perfect: An Investigation of SAM on Different Real-World Applications","volume":"21","author":"Ji","year":"2024","journal-title":"Mach. Intell. Res."},{"key":"ref_27","first-page":"1","article-title":"RingMo-SAM: A Foundation Model for Segment Anything in Multimodal Remote-Sensing Images","volume":"61","author":"Yan","year":"2023","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1109\/TGRS.2024.3477943","article-title":"RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation Based on Visual Foundation Model","volume":"62","author":"Chen","year":"2024","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_29","unstructured":"Nguyen, K.D., Phung, T.-H., and Cao, H.-G. (2023). A SAM-Based Solution for Hierarchical Panoptic Segmentation of Crops and Weeds Competition. arXiv."},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Cheng, B., Misra, I., Schwing, A.G., Kirillov, A., and Girdhar, R. (2022, January 19\u201320). Masked-Attention Mask Transformer for Universal Image Segmentation. Proceedings of the 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00135"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Li, Z., He, G., Fu, H., Chen, Q., Shangguan, B., Feng, P., and Jin, S. (2023, January 25\u201328). RS DINO: A Novel Panoptic Segmentation Algorithm for High Resolution Remote Sensing Images. Proceedings of the 2023 11th International Conference on Agro-Geoinformatics (Agro-Geoinformatics), Wuhan, China.","DOI":"10.1109\/Agro-Geoinformatics59224.2023.10233326"},{"key":"ref_32","doi-asserted-by":"crossref","unstructured":"Kirillov, A., Girshick, R., He, K., and Dollar, P. (2019, January 16\u201320). Panoptic Feature Pyramid Networks. Proceedings of the 2019 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00656"},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Li, Z., Wang, W., Xie, E., Yu, Z., Anandkumar, A., Alvarez, J.M., Luo, P., and Lu, T. (2022, January 18\u201324). Panoptic SegFormer: Delving Deeper into Panoptic Segmentation with Transformers. Proceedings of the 2022 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), New Orleans, LA, USA.","DOI":"10.1109\/CVPR52688.2022.00134"},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Li, F., Zhang, H., Xu, H., Liu, S., Zhang, L., Ni, L.M., and Shum, H.-Y. (2023, January 18\u201322). Mask DINO: Towards A Unified Transformer-Based Framework for Object Detection and Segmentation. Proceedings of the 2023 IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Vancouver, BC, Canada.","DOI":"10.1109\/CVPR52729.2023.00297"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep Residual Learning for Image Recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., and Guo, B. (2021, January 11\u201317). Swin Transformer: Hierarchical Vision Transformer Using Shifted Windows. Proceedings of the 2021 IEEE\/CVF International Conference on Computer Vision (ICCV), Montreal, QC, Canada.","DOI":"10.1109\/ICCV48922.2021.00986"},{"key":"ref_37","unstructured":"Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G., and Gelly, S. (2020). An Image Is Worth 16 \u00d7 16 Words: Transformers for Image Recognition at Scale. arXiv."},{"key":"ref_38","first-page":"17864","article-title":"Per-Pixel Classification Is Not All You Need for Semantic Segmentation","volume":"Volume 34","author":"Ranzato","year":"2021","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"}],"container-title":["Remote Sensing"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/21\/4002\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T16:22:11Z","timestamp":1760113331000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2072-4292\/16\/21\/4002"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,10,28]]},"references-count":38,"journal-issue":{"issue":"21","published-online":{"date-parts":[[2024,11]]}},"alternative-id":["rs16214002"],"URL":"https:\/\/doi.org\/10.3390\/rs16214002","relation":{},"ISSN":["2072-4292"],"issn-type":[{"value":"2072-4292","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,10,28]]}}}