{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T03:25:03Z","timestamp":1760239503018,"version":"build-2065373602"},"reference-count":47,"publisher":"MDPI AG","issue":"23","license":[{"start":{"date-parts":[[2020,11,25]],"date-time":"2020-11-25T00:00:00Z","timestamp":1606262400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"publisher","award":["Grant 61971095, Grant 61871078, Grant 61831005 and Grant 61871087."],"award-info":[{"award-number":["Grant 61971095, Grant 61871078, Grant 61831005 and Grant 61871087."]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Removing raindrops from a single image is a challenging problem due to the complex changes in shape, scale, and transparency among raindrops. Previous explorations have mainly been limited in two ways. First, publicly available raindrop image datasets have limited capacity in terms of modeling raindrop characteristics (e.g., raindrop collision and fusion) in real-world scenes. Second, recent deraining methods tend to apply shape-invariant filters to cope with diverse rainy images and fail to remove raindrops that are especially varied in shape and scale. In this paper, we address these raindrop removal problems from two perspectives. First, we establish a large-scale dataset named RaindropCityscapes, which includes 11,583 pairs of raindrop and raindrop-free images, covering a wide variety of raindrops and background scenarios. Second, a two-branch Multi-scale Shape Adaptive Network (MSANet) is proposed to detect and remove diverse raindrops, effectively filtering the occluded raindrop regions and keeping the clean background well-preserved. Extensive experiments on synthetic and real-world datasets demonstrate that the proposed method achieves significant improvements over the recent state-of-the-art raindrop removal methods. Moreover, the extension of our method towards the rainy image segmentation and detection tasks validates the practicality of the proposed method in outdoor applications.<\/jats:p>","DOI":"10.3390\/s20236733","type":"journal-article","created":{"date-parts":[[2020,11,25]],"date-time":"2020-11-25T08:59:15Z","timestamp":1606294755000},"page":"6733","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":3,"title":["Multi-Scale Shape Adaptive Network for Raindrop Detection and Removal from a Single Image"],"prefix":"10.3390","volume":"20","author":[{"given":"Hao","family":"Luo","sequence":"first","affiliation":[{"name":"School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2936-6340","authenticated-orcid":false,"given":"Qingbo","family":"Wu","sequence":"additional","affiliation":[{"name":"School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1946-3235","authenticated-orcid":false,"given":"King Ngi","family":"Ngan","sequence":"additional","affiliation":[{"name":"School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hanxiao","family":"Luo","sequence":"additional","affiliation":[{"name":"School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Haoran","family":"Wei","sequence":"additional","affiliation":[{"name":"School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Hongliang","family":"Li","sequence":"additional","affiliation":[{"name":"School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Fanman","family":"Meng","sequence":"additional","affiliation":[{"name":"School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Linfeng","family":"Xu","sequence":"additional","affiliation":[{"name":"School of Information and Communication Engineering, University of Electronic Science and Technology of China, Chengdu 611731, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2020,11,25]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","unstructured":"Zhu, C., and Yin, X.C. (2019). Detecting multi-resolution pedestrians using group cost-sensitive boosting with channel features. Sensors, 19.","DOI":"10.3390\/s19040780"},{"key":"ref_2","doi-asserted-by":"crossref","unstructured":"Arshad, S., Sualeh, M., Kim, D., Nam, D.V., and Kim, G.W. (2020). Clothoid: an integrated hierarchical framework for autonomous driving in a dynamic urban environment. Sensors, 20.","DOI":"10.3390\/s20185053"},{"key":"ref_3","unstructured":"Sindagi, V.A., and Patel, V.M. (November, January 27). Multi-level bottom-top and top-bottom feature fusion for crowd counting. Proceedings of the IEEE International Conference on Computer Vision, Seoul, Korea."},{"key":"ref_4","doi-asserted-by":"crossref","unstructured":"Sun, R., Huang, Q., Xia, M., and Zhang, J. (2018). Video-based person re-identification by an end-to-end learning architecture with hybrid deep appearance-temporal feature. Sensors, 18.","DOI":"10.3390\/s18113669"},{"key":"ref_5","doi-asserted-by":"crossref","unstructured":"Li, X., Wu, J., Lin, Z., Liu, H., and Zha, H. (2018, January 8\u201314). Recurrent squeeze-and-excitation context aggregation net for single image deraining. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_16"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2944","DOI":"10.1109\/TIP.2017.2691802","article-title":"Clearing the skies: A deep network architecture for single-image rain removal","volume":"26","author":"Fu","year":"2017","journal-title":"IEEE Trans. Image Process."},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Zhang, H., and Patel, V.M. (2018, January 18\u201322). Density-aware single image de-raining using a multi-stream dense network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00079"},{"key":"ref_8","doi-asserted-by":"crossref","unstructured":"Wang, T., Yang, X., Xu, K., Chen, S., Zhang, Q., and Lau, R.W. (2019, January 16\u201320). Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.01255"},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"84420","DOI":"10.1109\/ACCESS.2019.2922549","article-title":"From coarse to fine: A stage-wise deraining net","volume":"7","author":"Wang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_10","doi-asserted-by":"crossref","unstructured":"Ren, Y., Li, S., Nie, M., and Li, C. (2020). Single Image De-Raining via Improved Generative Adversarial Nets. Sensors, 20.","DOI":"10.3390\/s20061591"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Li, R., Cheong, L.F., and Tan, R.T. (2019, January 16\u201320). Heavy Rain Image Restoration: Integrating Physics Model and Conditional Adversarial Learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00173"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Hu, X., Fu, C.W., Zhu, L., and Heng, P.A. (2019, January 16\u201320). Depth-attentional Features for Single-image Rain Removal. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00821"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"963","DOI":"10.1016\/j.cag.2013.08.004","article-title":"A heuristic approach to the simulation of water drops and flows on glass panes","volume":"37","author":"Chen","year":"2013","journal-title":"Comput. Graph."},{"key":"ref_14","doi-asserted-by":"crossref","unstructured":"Fu, X., Huang, J., Zeng, D., Huang, Y., Ding, X., and Paisley, J. (2017, January 21\u201326). Removing rain from single images via a deep detail network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.186"},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Roser, M., and Geiger, A. (October, January 27). Video-based raindrop detection for improved image registration. Proceedings of the 2009 IEEE 12th International Conference on Computer Vision Workshops (ICCV Workshops), Kyoto, Japan.","DOI":"10.1109\/ICCVW.2009.5457650"},{"key":"ref_16","doi-asserted-by":"crossref","unstructured":"You, S., Tan, R.T., Kawakami, R., and Ikeuchi, K. (2013, January 23\u201328). Adherent raindrop detection and removal in video. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Portland, OR, USA.","DOI":"10.1109\/CVPR.2013.138"},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"You, S., Tan, R.T., Kawakami, R., Mukaigawa, Y., and Ikeuchi, K. (2014, January 1\u20135). Raindrop detection and removal from long range trajectories. Proceedings of the Asian Conference on Computer Vision, Singapore.","DOI":"10.1007\/978-3-319-16808-1_38"},{"key":"ref_18","doi-asserted-by":"crossref","unstructured":"Eigen, D., Krishnan, D., and Fergus, R. (2013, January 1\u20138). Restoring an image taken through a window covered with dirt or rain. Proceedings of the IEEE International Conference on Computer Vision, Sydney, Australia.","DOI":"10.1109\/ICCV.2013.84"},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Voulodimos, A., Doulamis, N., Doulamis, A., and Protopapadakis, E. (2018). Deep learning for computer vision: A brief review. Comput. Intell. Neurosci., 2018.","DOI":"10.1155\/2018\/7068349"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1016\/j.neucom.2015.09.116","article-title":"Deep learning for visual understanding: A review","volume":"187","author":"Guo","year":"2016","journal-title":"Neurocomputing"},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Qian, R., Tan, R.T., Yang, W., Su, J., and Liu, J. (2018, January 18\u201322). Attentive generative adversarial network for raindrop removal from a single image. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, UT, USA.","DOI":"10.1109\/CVPR.2018.00263"},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"Wu, Q., Zhang, W., and Kumar, B.V. (October, January 30). Raindrop detection and removal using salient visual features. Proceedings of the 2012 19th IEEE International Conference on Image Processing, Orlando, FL, USA.","DOI":"10.1109\/ICIP.2012.6467016"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"83","DOI":"10.1109\/TMM.2013.2284759","article-title":"Self-learning based image decomposition with applications to single image denoising","volume":"16","author":"Huang","year":"2013","journal-title":"IEEE Trans. Multimed."},{"key":"ref_24","unstructured":"Li, Y., Tan, R.T., Guo, X., Lu, J., and Brown, M.S. (July, January 26). Rain streak removal using layer priors. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_25","doi-asserted-by":"crossref","unstructured":"Luo, Y., Xu, Y., and Ji, H. (2015, January 7\u201313). Removing rain from a single image via discriminative sparse coding. Proceedings of the IEEE International Conference on Computer Vision, Santiago, Chile.","DOI":"10.1109\/ICCV.2015.388"},{"key":"ref_26","doi-asserted-by":"crossref","unstructured":"Zhu, L., Fu, C.W., Lischinski, D., and Heng, P.A. (2017, January 22\u201329). Joint bi-layer optimization for single-image rain streak removal. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.276"},{"key":"ref_27","doi-asserted-by":"crossref","unstructured":"Dai, J., Qi, H., Xiong, Y., Li, Y., Zhang, G., Hu, H., and Wei, Y. (2017, January 22\u201329). Deformable convolutional networks. Proceedings of the IEEE International Conference on Computer Vision, Venice, Italy.","DOI":"10.1109\/ICCV.2017.89"},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Zhang, C., and Kim, J. (2019, January 16\u201320). Object detection with location-aware deformable convolution and backward attention filtering. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00968"},{"key":"ref_29","doi-asserted-by":"crossref","unstructured":"Guo, D., Li, K., Zha, Z.J., and Wang, M. (2019, January 21\u201325). Dadnet: Dilated-attention-deformable convnet for crowd counting. Proceedings of the 27th ACM International Conference on Multimedia, Nice, France.","DOI":"10.1145\/3343031.3350881"},{"key":"ref_30","doi-asserted-by":"crossref","unstructured":"Wang, X., Chan, K.C., Yu, K., Dong, C., and Change Loy, C. (2019, January 16\u201317). Edvr: Video restoration with enhanced deformable convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA.","DOI":"10.1109\/CVPRW.2019.00247"},{"key":"ref_31","doi-asserted-by":"crossref","unstructured":"Tian, Y., Zhang, Y., Fu, Y., and Xu, C. (2020, January 14\u201319). TDAN: Temporally-Deformable Alignment Network for Video Super-Resolution. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00342"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"177734","DOI":"10.1109\/ACCESS.2019.2958030","article-title":"Deformable Non-Local Network for Video Super-Resolution","volume":"7","author":"Wang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_33","unstructured":"Cordts, M., Omran, M., Ramos, S., Rehfeld, T., Enzweiler, M., Benenson, R., Franke, U., Roth, S., and Schiele, B. (July, January 26). The cityscapes dataset for semantic urban scene understanding. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Liu, S., Huang, D., and Wang, Y. (2018, January 8\u201314). Receptive field block net for accurate and fast object detection. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01252-6_24"},{"key":"ref_35","doi-asserted-by":"crossref","unstructured":"Wu, Z., Su, L., and Huang, Q. (2019, January 16\u201320). Cascaded partial decoder for fast and accurate salient object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00403"},{"key":"ref_36","doi-asserted-by":"crossref","unstructured":"Zhu, X., Hu, H., Lin, S., and Dai, J. (2019, January 16\u201320). Deformable convnets v2: More deformable, better results. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00953"},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Huang, G., Liu, Z., Van Der Maaten, L., and Weinberger, K.Q. (2017, January 21\u201326). Densely connected convolutional networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.243"},{"key":"ref_38","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (July, January 26). Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, NV, USA."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"600","DOI":"10.1109\/TIP.2003.819861","article-title":"Image quality assessment: From error visibility to structural similarity","volume":"13","author":"Wang","year":"2004","journal-title":"IEEE Trans. Image Process."},{"key":"ref_40","unstructured":"Paszke, A., Gross, S., Massa, F., Lerer, A., Bradbury, J., Chanan, G., Killeen, T., Lin, Z., Gimelshein, N., and Antiga, L. (2019, January 8\u201314). Pytorch: An imperative style, high-performance deep learning library. Proceedings of the Advances in Neural Information Processing Systems 33 (NIPS 2019), Vancouver, BC, Canada."},{"key":"ref_41","unstructured":"Kingma, D.P., and Ba, J. (2014). Adam: A method for stochastic optimization. arXiv."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Isola, P., Zhu, J.Y., Zhou, T., and Efros, A.A. (2017, January 21\u201326). Image-to-image translation with conditional adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.632"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Tang, H., Xu, D., Sebe, N., Wang, Y., Corso, J.J., and Yan, Y. (2019, January 16\u201320). Multi-channel attention selection gan with cascaded semantic guidance for cross-view image translation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, CA, USA.","DOI":"10.1109\/CVPR.2019.00252"},{"key":"ref_44","doi-asserted-by":"crossref","first-page":"800","DOI":"10.1049\/el:20080522","article-title":"Scope of validity of PSNR in image\/video quality assessment","volume":"44","author":"Ghanbari","year":"2008","journal-title":"Electron. Lett."},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Jiang, K., Wang, Z., Yi, P., Chen, C., Huang, B., Luo, Y., Ma, J., and Jiang, J. (2020, January 14\u201319). Multi-scale progressive fusion network for single image deraining. Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA.","DOI":"10.1109\/CVPR42600.2020.00837"},{"key":"ref_46","doi-asserted-by":"crossref","unstructured":"Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21\u201326). Pyramid scene parsing network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.660"},{"key":"ref_47","unstructured":"Ren, S., He, K., Girshick, R., and Sun, J. (2015, January 7\u201312). Faster r-cnn: Towards real-time object detection with region proposal networks. Proceedings of the Advances in Neural Information Processing Systems 28 (NIPS 2015), Montreal, QC, Canada."}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/23\/6733\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T10:37:04Z","timestamp":1760179024000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/20\/23\/6733"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,11,25]]},"references-count":47,"journal-issue":{"issue":"23","published-online":{"date-parts":[[2020,12]]}},"alternative-id":["s20236733"],"URL":"https:\/\/doi.org\/10.3390\/s20236733","relation":{},"ISSN":["1424-8220"],"issn-type":[{"type":"electronic","value":"1424-8220"}],"subject":[],"published":{"date-parts":[[2020,11,25]]}}}