{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,28]],"date-time":"2026-04-28T21:34:14Z","timestamp":1777412054205,"version":"3.51.4"},"reference-count":82,"publisher":"MDPI AG","issue":"5","license":[{"start":{"date-parts":[[2024,2,26]],"date-time":"2024-02-26T00:00:00Z","timestamp":1708905600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"funder":[{"name":"Bohai Rim Energy Research Institute of Northeast Petroleum University","award":["HBHZX202002"],"award-info":[{"award-number":["HBHZX202002"]}]},{"name":"Bohai Rim Energy Research Institute of Northeast Petroleum University","award":["KYCXTD201903"],"award-info":[{"award-number":["KYCXTD201903"]}]},{"name":"Bohai Rim Energy Research Institute of Northeast Petroleum University","award":["SJGY20200125"],"award-info":[{"award-number":["SJGY20200125"]}]},{"name":"Bohai Rim Energy Research Institute of Northeast Petroleum University","award":["2022YFC330160204"],"award-info":[{"award-number":["2022YFC330160204"]}]},{"name":"project of Excellent and Middle-aged Scientific Research Innovatehicion Team of Northeast Petroleum University","award":["HBHZX202002"],"award-info":[{"award-number":["HBHZX202002"]}]},{"name":"project of Excellent and Middle-aged Scientific Research Innovatehicion Team of Northeast Petroleum University","award":["KYCXTD201903"],"award-info":[{"award-number":["KYCXTD201903"]}]},{"name":"project of Excellent and Middle-aged Scientific Research Innovatehicion Team of Northeast Petroleum University","award":["SJGY20200125"],"award-info":[{"award-number":["SJGY20200125"]}]},{"name":"project of Excellent and Middle-aged Scientific Research Innovatehicion Team of Northeast Petroleum University","award":["2022YFC330160204"],"award-info":[{"award-number":["2022YFC330160204"]}]},{"name":"Heilongjiang Province Higher Education Teaching Reform Project","award":["HBHZX202002"],"award-info":[{"award-number":["HBHZX202002"]}]},{"name":"Heilongjiang Province Higher Education Teaching Reform Project","award":["KYCXTD201903"],"award-info":[{"award-number":["KYCXTD201903"]}]},{"name":"Heilongjiang Province Higher Education Teaching Reform Project","award":["SJGY20200125"],"award-info":[{"award-number":["SJGY20200125"]}]},{"name":"Heilongjiang Province Higher Education Teaching Reform Project","award":["2022YFC330160204"],"award-info":[{"award-number":["2022YFC330160204"]}]},{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["HBHZX202002"],"award-info":[{"award-number":["HBHZX202002"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["KYCXTD201903"],"award-info":[{"award-number":["KYCXTD201903"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["SJGY20200125"],"award-info":[{"award-number":["SJGY20200125"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100012166","name":"National Key Research and Development Program of China","doi-asserted-by":"publisher","award":["2022YFC330160204"],"award-info":[{"award-number":["2022YFC330160204"]}],"id":[{"id":"10.13039\/501100012166","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Remote sensing images change detection technology has become a popular tool for monitoring the change type, area, and distribution of land cover, including cultivated land, forest land, photovoltaic, roads, and buildings. However, traditional methods which rely on pre-annotation and on-site verification are time-consuming and challenging to meet timeliness requirements. With the emergence of artificial intelligence, this paper proposes an automatic change detection model and a crowdsourcing collaborative framework. The framework uses human-in-the-loop technology and an active learning approach to transform the manual interpretation method into a human-machine collaborative intelligent interpretation method. This low-cost and high-efficiency framework aims to solve the problem of weak model generalization caused by the lack of annotated data in change detection. The proposed framework can effectively incorporate expert domain knowledge and reduce the cost of data annotation while improving model performance. To ensure data quality, a crowdsourcing quality control model is constructed to evaluate the annotation qualification of the annotators and check their annotation results. Furthermore, a prototype of automatic detection and crowdsourcing collaborative annotation management platform is developed, which integrates annotation, crowdsourcing quality control, and change detection applications. The proposed framework and platform can help natural resource departments monitor land cover changes efficiently and effectively.<\/jats:p>","DOI":"10.3390\/s24051509","type":"journal-article","created":{"date-parts":[[2024,2,26]],"date-time":"2024-02-26T06:50:23Z","timestamp":1708930223000},"page":"1509","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":10,"title":["A Deep Learning Based Platform for Remote Sensing Images Change Detection Integrating Crowdsourcing and Active Learning"],"prefix":"10.3390","volume":"24","author":[{"given":"Zhibao","family":"Wang","sequence":"first","affiliation":[{"name":"School of Computer and Information Technology, Northeast Petroleum University, Daqing 163318, China"},{"name":"Bohai-Rim Energy Research Institute, Northeast Petroleum University, Qinhuangdao 066004, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jie","family":"Zhang","sequence":"additional","affiliation":[{"name":"School of Computer and Information Technology, Northeast Petroleum University, Daqing 163318, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1242-5412","authenticated-orcid":false,"given":"Lu","family":"Bai","sequence":"additional","affiliation":[{"name":"School of Electronics, Electrical Engineering and Computer Science, Queen\u2019s University Belfast, Belfast BT9 6SB, UK"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Huan","family":"Chang","sequence":"additional","affiliation":[{"name":"School of Computer and Information Technology, Northeast Petroleum University, Daqing 163318, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9245-1102","authenticated-orcid":false,"given":"Yuanlin","family":"Chen","sequence":"additional","affiliation":[{"name":"School of Computer and Information Technology, Northeast Petroleum University, Daqing 163318, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Ying","family":"Zhang","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Remote Sensing Science, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing Normal University, Beijing 100101, China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Jinhua","family":"Tao","sequence":"additional","affiliation":[{"name":"State Key Laboratory of Remote Sensing Science, Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing Normal University, Beijing 100101, China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2024,2,26]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"115","DOI":"10.1016\/j.isprsjprs.2014.03.009","article-title":"Who launched what, when and why; trends in global land-cover observation capacity from civilian earth observation satellites","volume":"103","author":"Belward","year":"2015","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"193","DOI":"10.4236\/ars.2013.22022","article-title":"Investigating the use of remote sensing and GIS techniques to detect land use and land cover change: A review","volume":"2","author":"Alqurashi","year":"2013","journal-title":"Adv. Remote Sens."},{"key":"ref_3","unstructured":"Hecheltjen, A., Thonfeld, F., and Menz, G. (2014). Land Use and Land Cover Mapping in Europe: Practices & Trends, Springer."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"571","DOI":"10.1016\/j.proenv.2010.10.062","article-title":"Analysis of the impact of land use\/land cover change on land surface temperature with remote sensing","volume":"2","author":"Jiang","year":"2010","journal-title":"Procedia Environ. Sci."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"611","DOI":"10.4236\/ijg.2017.84033","article-title":"Accuracy assessment of land use\/land cover classification using remote sensing and GIS","volume":"8","author":"Rwanga","year":"2017","journal-title":"Int. J. Geosci."},{"key":"ref_6","doi-asserted-by":"crossref","unstructured":"Vali, A., Comai, S., and Matteucci, M. (2020). Deep learning for land use and land cover classification based on hyperspectral and multispectral earth observation data: A review. Remote Sens., 12.","DOI":"10.3390\/rs12152495"},{"key":"ref_7","doi-asserted-by":"crossref","unstructured":"Zhang, X., Han, L., Han, L., and Zhu, L. (2020). How well do deep learning-based methods for land cover classification and object detection perform on high resolution remote sensing imagery?. Remote Sens., 12.","DOI":"10.3390\/rs12030417"},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"18","DOI":"10.1016\/j.ijdrr.2018.11.022","article-title":"Review of flood disaster studies in Nepal: A remote sensing perspective","volume":"34","author":"Zhang","year":"2019","journal-title":"Int. J. Disaster Risk Reduct."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1016\/j.isprsjprs.2005.02.002","article-title":"Satellite remote sensing of earthquake, volcano, flood, landslide and coastal inundation hazards","volume":"59","author":"Tralli","year":"2005","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"11","DOI":"10.1016\/j.isprsjprs.2016.03.014","article-title":"A survey on object detection in optical remote sensing images","volume":"117","author":"Cheng","year":"2016","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Shi, K., Bai, L., Wang, Z., Tong, X., Mulvenna, M.D., and Bond, R.R. (2022, January 17\u201322). Photovoltaic Installations Change Detection from Remote Sensing Images Using Deep Learning. Proceedings of the IGARSS 2022\u20132022 IEEE International Geoscience and Remote Sensing Symposium, Kuala Lumpur, Malaysia.","DOI":"10.1109\/IGARSS46834.2022.9883738"},{"key":"ref_12","doi-asserted-by":"crossref","unstructured":"Wang, Z., Bai, L., Song, G., Zhang, J., Tao, J., Mulvenna, M.D., Bond, R.R., and Chen, L. (2021). An oil well dataset derived from satellite-based remote sensing. Remote Sens., 13.","DOI":"10.3390\/rs13061132"},{"key":"ref_13","doi-asserted-by":"crossref","unstructured":"Song, G., Wang, Z., Bai, L., Zhang, J., and Chen, L. (2020, January 21\u201325). Detection of oil wells based on faster R-CNN in optical satellite remote sensing images. Proceedings of the Image Signal Processing for Remote Sensing XXVI, Online Only.","DOI":"10.1117\/12.2572996"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"574","DOI":"10.1109\/TGRS.2018.2858817","article-title":"Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set","volume":"57","author":"Ji","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_15","doi-asserted-by":"crossref","unstructured":"Daudt, R.C., Le Saux, B., Boulch, A., and Gousseau, Y. (2018, January 22\u201327). Urban change detection for multispectral earth observation using convolutional neural networks. Proceedings of the IGARSS 2018\u20142018 IEEE International Geoscience and Remote Sensing Symposium, Valencia, Spain.","DOI":"10.1109\/IGARSS.2018.8518015"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"102783","DOI":"10.1016\/j.cviu.2019.07.003","article-title":"Multitask learning for large-scale semantic change detection","volume":"187","author":"Daudt","year":"2019","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_17","doi-asserted-by":"crossref","unstructured":"Li, H., Dou, X., Tao, C., Wu, Z., Chen, J., Peng, J., Deng, M., and Zhao, L. (2020). RSI-CB: A large-scale remote sensing image classification benchmark using crowdsourced data. Sensors, 20.","DOI":"10.3390\/s20061594"},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"52","DOI":"10.1016\/j.autcon.2018.10.019","article-title":"A survey of automation-enabled human-in-the-loop systems for infrastructure visual inspection","volume":"97","author":"Agnisarman","year":"2019","journal-title":"Autom. Constr."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"606","DOI":"10.1109\/JSTSP.2011.2139193","article-title":"A survey of active learning algorithms for supervised remote sensing image classification","volume":"5","author":"Tuia","year":"2011","journal-title":"IEEE J. Sel. Top. Signal Process."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"4057","DOI":"10.1080\/01431161.2020.1714774","article-title":"Using convolutional neural networks incorporating hierarchical active learning for target-searching in large-scale remote sensing images","volume":"41","author":"Xu","year":"2020","journal-title":"Int. J. Remote Sens."},{"key":"ref_21","doi-asserted-by":"crossref","unstructured":"Jiang, H., Peng, M., Zhong, Y., Xie, H., Hao, Z., Lin, J., Ma, X., and Hu, X. (2022). A Survey on Deep Learning-Based Change Detection from High-Resolution Remote Sensing Images. Remote Sens., 14.","DOI":"10.3390\/rs14071552"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"13680","DOI":"10.1109\/JSEN.2023.3271391","article-title":"Cross-Attention Guided Group Aggregation Network for Cropland Change Detection","volume":"23","author":"Xu","year":"2023","journal-title":"IEEE Sens. J."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"237","DOI":"10.1109\/TGRS.2007.907973","article-title":"Hyperspectral Change Detection in the Presenceof Diurnal and Seasonal Variations","volume":"46","author":"Eismann","year":"2007","journal-title":"IEEE Trans. Geosci. Remote. Sens."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Zhang, J., Wang, Z., Bai, L., Song, G., Tao, J., and Chen, L. (2021, January 11\u201316). Deforestation Detection Based on U-Net and LSTM in Optical Satellite Remote Sensing Images. Proceedings of the 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS, Brussels, Belgium.","DOI":"10.1109\/IGARSS47720.2021.9554689"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"143","DOI":"10.1007\/s12145-019-00380-5","article-title":"Change detection techniques for remote sensing applications: A survey","volume":"12","author":"Asokan","year":"2019","journal-title":"Earth Sci. Inform."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"294","DOI":"10.1109\/TIP.2004.838698","article-title":"Image change detection algorithms: A systematic survey","volume":"14","author":"Radke","year":"2005","journal-title":"IEEE Trans. Image Process."},{"key":"ref_27","unstructured":"Murat, \u0130., and \u00dcnsalan, C. (2012). Two-Dimensional Change Detection Methods: Remote Sensing Applications, Springer Science & Business Media."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Kwan, C. (2019). Methods and challenges using multispectral and hyperspectral images for practical change detection applications. Information, 10.","DOI":"10.3390\/info10110353"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"27442","DOI":"10.1109\/ACCESS.2018.2807380","article-title":"Adaptive Change Detection with Significance Test","volume":"6","author":"Ke","year":"2018","journal-title":"IEEE Access"},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"1","DOI":"10.17485\/ijst\/2016\/v9i42\/99682","article-title":"A novel multiple unsupervised algorithm for land use\/land cover classification","volume":"9","author":"Vignesh","year":"2016","journal-title":"Indian J. Sci. Technol."},{"key":"ref_31","doi-asserted-by":"crossref","first-page":"376","DOI":"10.1007\/s10661-018-6751-y","article-title":"Object-based land-use\/land-cover change detection using Landsat imagery: A case study of Ardabil, Namin, and Nir counties in northwest Iran","volume":"190","author":"Aslami","year":"2018","journal-title":"Environ. Monit. Assess."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"2151","DOI":"10.1109\/JSTARS.2015.2427274","article-title":"Unsupervised Change Detection in Multitemporal Multispectral Satellite Images Using Parallel Particle Swarm Optimization","volume":"8","author":"Kusetogullari","year":"2015","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote. Sens."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"225","DOI":"10.5721\/EuJRS20164913","article-title":"Unsupervised change detection in high spatial resolution remote sensing images based on a conditional random field model","volume":"49","author":"Cao","year":"2016","journal-title":"Eur. J. Remote. Sens."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"Ajadi, O.A., Meyer, F.J., and Webley, P.W. (2016). Change Detection in Synthetic Aperture Radar Images Using a Multiscale-Driven Approach. Remote. Sens., 8.","DOI":"10.3390\/rs8060482"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"414","DOI":"10.1109\/LGRS.2016.2645742","article-title":"Change Detection in Multispectral Landsat Images Using Multiobjective Evolutionary Algorithm","volume":"14","author":"Yavariabdi","year":"2017","journal-title":"IEEE Geosci. Remote. Sens. Lett."},{"key":"ref_36","first-page":"8005205","article-title":"Change detection in image time-series using unsupervised lstm","volume":"19","author":"Saha","year":"2020","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_37","doi-asserted-by":"crossref","unstructured":"Sefrin, O., Riese, F.M., and Keller, S. (2020). Deep learning for land cover change detection. Remote Sens., 13.","DOI":"10.3390\/rs13010078"},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Navab, N., Hornegger, J., Wells, W.M., and Frangi, A.F. (2015). Medical Image Computing and Computer-Assisted Intervention\u2014MICCAI 2015, Lecture Notes in Computer Science, Springer International Publishing.","DOI":"10.1007\/978-3-319-24571-3"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"2504905","DOI":"10.1109\/LGRS.2022.3173300","article-title":"Simple multiscale unet for change detection with heterogeneous remote sensing images","volume":"19","author":"Lv","year":"2022","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"028505","DOI":"10.1117\/1.JRS.15.028505","article-title":"Hyperspectral change detection based on modification of UNet neural networks","volume":"15","author":"Moustafa","year":"2021","journal-title":"J. Appl. Remote Sens."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Peng, D., Zhang, Y., and Guan, H. (2019). End-to-End Change Detection for High Resolution Satellite Images Using Improved UNet++. Remote Sens., 11.","DOI":"10.3390\/rs11111382"},{"key":"ref_42","unstructured":"Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., and Liang, J. (2018). Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support, 2018 4th International Workshop, DLMIA 2018, and 8th International Workshop, ML-CDS 2018, Held in Conjunction with MICCAI 2018, Granada, Spain, 20 September 2018, Springer. Proceedings 4."},{"key":"ref_43","first-page":"8006605","article-title":"DifUnet++: A Satellite Images Change Detection Network Based on Unet++ and Differential Pyramid","volume":"19","author":"Zhang","year":"2022","journal-title":"IEEE Geosci. Remote Sens. Lett"},{"key":"ref_44","doi-asserted-by":"crossref","unstructured":"Chaurasia, A., and Culurciello, E. (2018, January 10\u201313). LinkNet: Exploiting encoder representations for efficient semantic segmentation. Proceedings of the 2017 IEEE Visual Communications and Image Processing (VCIP), St. Petersburg, FL, USA.","DOI":"10.1109\/VCIP.2017.8305148"},{"key":"ref_45","doi-asserted-by":"crossref","unstructured":"Zhou, L., Zhang, C., and Wu, M. (2018, January 18\u201322). D-LinkNet: LinkNet with Pretrained Encoder and Dilated Convolution for High Resolution Satellite Imagery Road Extraction. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPRW.2018.00034"},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"811","DOI":"10.1109\/LGRS.2020.2988032","article-title":"Building Change Detection for Remote Sensing Images Using a Dual-Task Constrained Deep Siamese Convolutional Network Model","volume":"18","author":"Liu","year":"2021","journal-title":"IEEE Geosci. Remote Sens. Lett"},{"key":"ref_47","doi-asserted-by":"crossref","unstructured":"Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21\u201326). Pyramid scene parsing network. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.660"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Furukawa, Y., Suzuki, K., Hamaguchi, R., Onishi, M., and Sakurada, K. (2020\u201424, January 24). Self-supervised Simultaneous Alignment and Change Detection, in: IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS). Proceedings of the 2020 IEEE\/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NV, USA.","DOI":"10.1109\/IROS45743.2020.9340840"},{"key":"ref_49","doi-asserted-by":"crossref","unstructured":"Lin, T.Y., Doll\u00e1r, P., Girshick, R., He, K., Hariharan, B., and Belongie, S. (2017, January 21\u201326). Feature pyramid networks for object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.106"},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"2142","DOI":"10.1109\/JSTARS.2022.3152775","article-title":"ForkNet: Strong Semantic Feature Representation and Subregion Supervision for Accurate Remote Sensing Change Detection","volume":"15","author":"He","year":"2022","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_51","doi-asserted-by":"crossref","unstructured":"Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., and Adam, H. (2018, January 8\u201314). Encoder-Decoder with Atrous Separable Convolution for Semantic Image Segmentation. Proceedings of the European Conference on Computer Vision (ECCV), Munich, Germany.","DOI":"10.1007\/978-3-030-01234-2_49"},{"key":"ref_52","doi-asserted-by":"crossref","unstructured":"Chen, D., Ye, B., Zhao, Z., Wang, F., Xu, W., and Yin, W. (2022, January 18\u201322). Change Detection Converter: Using Semantic Segmantation Models to Tackle Change Detection Task. Proceedings of the 2022 IEEE International Conference on Multimedia and Expo (ICME), Taipei, Taiwan.","DOI":"10.1109\/ICME52920.2022.9859973"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"232","DOI":"10.1080\/20964471.2019.1657720","article-title":"A survey of remote sensing image classification based on CNNs","volume":"3","author":"Song","year":"2019","journal-title":"Big Earth Data"},{"key":"ref_54","doi-asserted-by":"crossref","unstructured":"Hu, Z., and Wu, W. (2012, January 8\u201312). A satellite data portal developed for crowdsourcing data analysis and interpretation. Proceedings of the 2012 IEEE 8th International Conference on E-Science (e-Science), Chicago, IL, USA.","DOI":"10.1109\/eScience.2012.6404453"},{"key":"ref_55","doi-asserted-by":"crossref","unstructured":"Irshad, H., Montaser-Kouhsari, L., Waltz, G., Bucur, O., Nowak, J.A., Dong, F., Knoblauch, N.W., and Beck, A.H. (2015, January 4\u20138). Crowdsourcing Image Annotation for Nucleus Detection and Segmentation in Computational Pathology: Evaluating Experts, Automated Methods, and the Crowd. Proceedings of the Pacific Symposium on Biocomputing 2015, World Scientific, Kohala Coast, HI, USA.","DOI":"10.1142\/9789814644730_0029"},{"key":"ref_56","doi-asserted-by":"crossref","first-page":"345","DOI":"10.3390\/rs1030345","article-title":"Geo-Wiki.Org: The Use of Crowdsourcing to Improve Global Land Cover","volume":"1","author":"Fritz","year":"2009","journal-title":"Remote Sens."},{"key":"ref_57","unstructured":"Foody, G., See, L., Fritz, S., Mooney, P., Olteanu-Raimond, A.-M., and Fonte, C.C. (2017). Mapping and the Citizen Sensor, Ubiquity Press."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Zhao, J., Wang, X., and Zhou, Y. (October, January 26). A Crowdsourcing-Based Platform for Labelling Remote Sensing Images. Proceedings of the IGARSS 2020\u20132020 IEEE International Geoscience and Remote Sensing Symposium, Waikoloa, HI, USA.","DOI":"10.1109\/IGARSS39084.2020.9323820"},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Franklin, M.J., Kossmann, D., Kraska, T., Ramesh, S., and Xin, R. (2011, January 12\u201316). CrowdDB: Answering queries with crowdsourcing. Proceedings of the 2011 ACM SIGMOD International Conference on Management of Data, Athens, Greece.","DOI":"10.1145\/1989323.1989331"},{"key":"ref_60","doi-asserted-by":"crossref","first-page":"711","DOI":"10.1007\/s00778-013-0328-8","article-title":"Hybrid entity clustering using crowds and data","volume":"22","author":"Lee","year":"2013","journal-title":"VLDB J."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"1990","DOI":"10.14778\/2367502.2367555","article-title":"Deco: A system for declarative crowdsourcing. Proc","volume":"5","author":"Park","year":"2012","journal-title":"VLDB Endow."},{"key":"ref_62","doi-asserted-by":"crossref","unstructured":"Yan, T., Kumar, V., and Ganesan, D. (2010, January 15\u201318). CrowdSearch: Exploiting crowds for accurate real-time image search on mobile phones. Proceedings of the 8th International Conference on Mobile Systems, Applications, and Services, San Francisco, CA, USA.","DOI":"10.1145\/1814433.1814443"},{"key":"ref_63","unstructured":"Alonso, O., and Mizzaro, S. (2009, January 23). Can we get rid of TREC assessors? Using Mechanical Turk for relevance assessment. Proceedings of the SIGIR 2009 Workshop on the Future of IR Evaluation 2009, Boston, MA, USA."},{"key":"ref_64","unstructured":"Yan, Y., Rosales, R., Fung, G., and Dy, J.G. (July, January 28). Active learning from crowds. Proceedings of the 28th International Conference on International Conference on Machine Learning, ICML\u201911, Madison, WI, USA."},{"key":"ref_65","doi-asserted-by":"crossref","first-page":"20","DOI":"10.2307\/2346806","article-title":"Maximum Likelihood Estimation of Observer Error-Rates Using the EM Algorithm","volume":"28","author":"Dawid","year":"1979","journal-title":"Appl. Stat."},{"key":"ref_66","unstructured":"Lee, D., Das Sarma, A., and Parameswaran, A. (2018). Aggregating crowdsourced image segmentations. HCOMP."},{"key":"ref_67","doi-asserted-by":"crossref","first-page":"1879","DOI":"10.1109\/TPDS.2015.2457924","article-title":"A Crowdsourcing Worker Quality Evaluation Algorithm on MapReduce for Big Data Applications","volume":"27","author":"Dang","year":"2015","journal-title":"IEEE Trans. Parallel Distrib. Syst."},{"key":"ref_68","unstructured":"Vuurens, J.B.P., and Eickhoff, C. (2011, January 28). How Much Spam Can You Take? An Analysis of Crowdsourcing Results to Increase Accuracy. Proceedings of the ACM SIGIR Workshop on Crowdsourcing for Information Retrieval (CIR\u201911), Beijing, China."},{"key":"ref_69","first-page":"37","article-title":"Human-in-the-loop Techniques in Machine Learning","volume":"43","author":"Chai","year":"2020","journal-title":"IEEE Data Eng. Bull."},{"key":"ref_70","doi-asserted-by":"crossref","first-page":"364","DOI":"10.1016\/j.future.2022.05.014","article-title":"A survey of human-in-the-loop for machine learning","volume":"135","author":"Wu","year":"2021","journal-title":"Future Gener. Comput. Syst."},{"key":"ref_71","doi-asserted-by":"crossref","unstructured":"Garc\u00eda Rodr\u00edguez, C., Vitri\u00e0, J., and Mora, O. (2020). Uncertainty-Based Human-in-the-Loop Deep Learning for Land Cover Segmentation. Remote Sens., 12.","DOI":"10.3390\/rs12223836"},{"key":"ref_72","doi-asserted-by":"crossref","first-page":"e2021EA002085","DOI":"10.1029\/2021EA002085","article-title":"Human-in-the-Loop Segmentation of Earth Surface Imagery","volume":"9","author":"Buscombe","year":"2022","journal-title":"Earth Space Sci."},{"key":"ref_73","doi-asserted-by":"crossref","unstructured":"Schulz, J., Buck, A., Anderson, D.T., Keller, J.M., Scott, G., and Luke, R.H. (2021, January 12\u201317). Human-in-the-loop extension to stream classification for labeling of low altitude drone imagery. Proceedings of the Autonomous Systems: Sensors, Processing, and Security for Vehicles and Infrastructure 2021, Online.","DOI":"10.1117\/12.2585851"},{"key":"ref_74","doi-asserted-by":"crossref","unstructured":"Wang, Z., Zhang, Z., Bai, L., Yang, Y., and Ma, Q. (2022, January 17\u201322). Application of an Improved U-Net Neural Network on Fracture Segmentation from Outcrop Images. Proceedings of the IGARSS 2020-2020 IEEE International Geoscience and Remote Sensing Symposium, Kuala Lumpur, Malaysia.","DOI":"10.1109\/IGARSS46834.2022.9883208"},{"key":"ref_75","unstructured":"Zhu, M., Wang, Z., Bai, L., Zhang, J., Tao, J., and Chen, L. (2021). Image and Signal Processing for Remote Sensing XXVII, SPIE."},{"key":"ref_76","first-page":"1","article-title":"A Survey on Active Deep Learning: From Model Driven to Data Driven","volume":"54","author":"Liu","year":"2022","journal-title":"ACM Comput. Surv"},{"key":"ref_77","doi-asserted-by":"crossref","first-page":"712","DOI":"10.1109\/JSTARS.2016.2598859","article-title":"Active Deep Learning for Classification of Hyperspectral Images","volume":"10","author":"Liu","year":"2017","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_78","unstructured":"R\u016f\u017ei\u010dka, V., D\u2019Aronco, S., Wegner, J.D., and Schindler, K. (2020). Deep Active Learning in Remote Sensing for data efficient Change Detection. arXiv."},{"key":"ref_79","doi-asserted-by":"crossref","first-page":"227","DOI":"10.1016\/0146-664X(80)90054-4","article-title":"Euclidean distance mapping","volume":"14","author":"Danielsson","year":"1980","journal-title":"Comput. Graph. Image Process."},{"key":"ref_80","doi-asserted-by":"crossref","first-page":"565","DOI":"10.5194\/isprs-archives-XLII-2-565-2018","article-title":"Change Detection in Remote Sensing Images Using Conditional Adversarial Networks","volume":"XLII-2","author":"Lebedev","year":"2018","journal-title":"ISPRS\u2014Int. Arch. Photogramm. Remote. Sens. Spat. Inf. Sci."},{"key":"ref_81","doi-asserted-by":"crossref","unstructured":"Chen, H., and Shi, Z. (2020). A Spatial-Temporal Attention-Based Method and a New Dataset for Remote Sensing Image Change Detection. Remote Sens., 12.","DOI":"10.3390\/rs12101662"},{"key":"ref_82","doi-asserted-by":"crossref","unstructured":"Garcia-Garcia, A., Orts-Escolano, S., Oprea, S., Villena-Martinez, V., and Garcia-Rodriguez, J. (2017). A Review on Deep Learning Techniques Applied to Semantic Segmentation. arXiv.","DOI":"10.1016\/j.asoc.2018.05.018"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/5\/1509\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,10]],"date-time":"2025-10-10T14:05:01Z","timestamp":1760105101000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/24\/5\/1509"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,2,26]]},"references-count":82,"journal-issue":{"issue":"5","published-online":{"date-parts":[[2024,3]]}},"alternative-id":["s24051509"],"URL":"https:\/\/doi.org\/10.3390\/s24051509","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2024,2,26]]}}}