{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,10,12]],"date-time":"2025-10-12T01:44:02Z","timestamp":1760233442686,"version":"build-2065373602"},"reference-count":42,"publisher":"MDPI AG","issue":"1","license":[{"start":{"date-parts":[[2021,1,13]],"date-time":"2021-01-13T00:00:00Z","timestamp":1610496000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["IJGI"],"abstract":"<jats:p>Earth Observation data possess tremendous potential in understanding the dynamics of our planet. We propose the Semantics-driven Remote Sensing Scene Understanding (Sem-RSSU) framework for rendering comprehensive grounded spatio-contextual scene descriptions for enhanced situational awareness. To minimize the semantic gap for remote-sensing-scene understanding, the framework puts forward the transformation of scenes by using semantic-web technologies to Remote Sensing Scene Knowledge Graphs (RSS-KGs). The knowledge-graph representation of scenes has been formalized through the development of a Remote Sensing Scene Ontology (RSSO)\u2014a core ontology for an inclusive remote-sensing-scene data product. The RSS-KGs are enriched both spatially and contextually, using a deductive reasoner, by mining for implicit spatio-contextual relationships between land-cover classes in the scenes. The Sem-RSSU, at its core, constitutes novel Ontology-driven Spatio-Contextual Triple Aggregation and realization algorithms to transform KGs to render grounded natural language scene descriptions. Considering the significance of scene understanding for informed decision-making from remote sensing scenes during a flood, we selected it as a test scenario, to demonstrate the utility of this framework. In that regard, a contextual domain knowledge encompassing Flood Scene Ontology (FSO) has been developed. Extensive experimental evaluations show promising results, further validating the efficacy of this framework.<\/jats:p>","DOI":"10.3390\/ijgi10010032","type":"journal-article","created":{"date-parts":[[2021,1,13]],"date-time":"2021-01-13T11:52:32Z","timestamp":1610538752000},"page":"32","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":13,"title":["Semantics-Driven Remote Sensing Scene Understanding Framework for Grounded Spatio-Contextual Scene Descriptions"],"prefix":"10.3390","volume":"10","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8168-857X","authenticated-orcid":false,"given":"Abhishek V.","family":"Potnis","sequence":"first","affiliation":[{"name":"Centre of Studies in Resources Engineering, Indian Institute of Technology Bombay, Mumbai, Maharashtra 400076, India"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Surya S.","family":"Durbha","sequence":"additional","affiliation":[{"name":"Centre of Studies in Resources Engineering, Indian Institute of Technology Bombay, Mumbai, Maharashtra 400076, India"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-9505-6204","authenticated-orcid":false,"given":"Rajat C.","family":"Shinde","sequence":"additional","affiliation":[{"name":"Centre of Studies in Resources Engineering, Indian Institute of Technology Bombay, Mumbai, Maharashtra 400076, India"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"1968","published-online":{"date-parts":[[2021,1,13]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"2563","DOI":"10.1109\/TGRS.2005.847908","article-title":"Semantics-enabled framework for knowledge discovery from Earth observation data archives","volume":"43","author":"Durbha","year":"2005","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_2","unstructured":"Datcu, M., Remote, G., and Data, S. (1996, January 31). Scene Understanding from SAR Images. Proceedings of the 1996 International Geoscience and Remote Sensing Symposium, Lincoln, NE, USA."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"1431","DOI":"10.1109\/36.718847","article-title":"Spatial information retrieval from remote-sensing images. I. Information theoretical perspective","volume":"36","author":"Datcu","year":"1998","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"1976","DOI":"10.1109\/TGRS.2003.814630","article-title":"Information fusion for scene understanding from interferometric SAR data in urban environments","volume":"41","author":"Quartulli","year":"2003","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_5","first-page":"669","article-title":"GeoBrowse: An integrated environment for satellite image retrieval and mining","volume":"2","author":"Marchisio","year":"1998","journal-title":"Int. Geosci. Remote Sens. Symp."},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"2923","DOI":"10.1109\/TGRS.2003.817197","article-title":"Information Mining in Remote Sensing Image Archives\u2014Part A: System Concepts","volume":"41","author":"Datcu","year":"2003","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"188","DOI":"10.1109\/TGRS.2004.838374","article-title":"Information mining in remote sensing image archives: System evaluation","volume":"43","author":"Daschiel","year":"2005","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"839","DOI":"10.1109\/TGRS.2006.890579","article-title":"GeoIRIS: Geospatial Information Retrieval and Indexing System\u2014Content Mining, Semantics Modeling, and Complex Queries","volume":"45","author":"Shyu","year":"2007","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"861","DOI":"10.1109\/TGRS.2006.890580","article-title":"Detecting Man-Made Structures and Changes in Satellite Imagery with a Content-Based Information Retrieval System Built on Self-Organizing Maps","volume":"45","author":"Molinier","year":"2007","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"29","DOI":"10.1109\/JSTARS.2016.2547992","article-title":"Semantics-Enabled Framework for Spatial Image Information Mining of Linked Earth Observation Data","volume":"10","author":"Kurte","year":"2017","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Qu, B., Li, X., Tao, D., and Lu, X. (2016, January 6\u20138). Deep semantic understanding of high resolution remote sensing image. Proceedings of the IEEE CITS 2016\u20142016 International Conference on Computer, Information and Telecommunication Systems (CITS), Kunming, China.","DOI":"10.1109\/CITS.2016.7546397"},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"3623","DOI":"10.1109\/TGRS.2017.2677464","article-title":"Can a Machine Generate Humanlike Language Descriptions for a Remote Sensing Image?","volume":"55","author":"ZPan","year":"2017","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"2183","DOI":"10.1109\/TGRS.2017.2776321","article-title":"Exploring Models and Data for Remote Sensing Image Caption Generation","volume":"56","author":"Lu","year":"2018","journal-title":"IEEE Trans. Geosci. Remote Sens."},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"1274","DOI":"10.1109\/LGRS.2019.2893772","article-title":"Semantic Descriptions of High-Resolution Remote Sensing Images","volume":"16","author":"Wang","year":"2019","journal-title":"IEEE Geosci. Remote Sens. Lett."},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"137355","DOI":"10.1109\/ACCESS.2019.2942154","article-title":"VAA: Visual Aligning Attention Model for Remote Sensing Image Captioning","volume":"7","author":"Zhang","year":"2019","journal-title":"IEEE Access"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"256","DOI":"10.1109\/JSTARS.2019.2959208","article-title":"Retrieval Topic Recurrent Memory Network for Remote Sensing Image Captioning","volume":"13","author":"Wang","year":"2020","journal-title":"IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"2608","DOI":"10.1109\/ACCESS.2019.2962195","article-title":"Exploring Multi-Level Attention and Semantic Relationship for Remote Sensing Image Captioning","volume":"8","author":"Yuan","year":"2020","journal-title":"IEEE Access"},{"key":"ref_18","unstructured":"Zhang, Y., Qiu, Z., Yao, T., Liu, D., and Mei, T. (2015, January 7\u201312). Fully Convolutional Adaptation Networks for Semantic Segmentation. Proceedings of the 2015 IEEE\/CVF Conference on Computer Vision and Pattern Recognition, Boston, MA, USA."},{"key":"ref_19","doi-asserted-by":"crossref","unstructured":"Ronneberger, O., Fischer, P., and Brox, T. (2015, January 5\u20139). U-Net: Convolutional Networks for Biomedical Image Segmentation. Proceedings of the International Conference on Medical Image Computing and Computer-Assisted Intervention, Munich, Germany. Lecture Notes in Computer Science.","DOI":"10.1007\/978-3-319-24574-4_28"},{"key":"ref_20","doi-asserted-by":"crossref","unstructured":"Zhao, H., Shi, J., Qi, X., Wang, X., and Jia, J. (2017, January 21\u201326). Pyramid Scene Parsing Network. Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA.","DOI":"10.1109\/CVPR.2017.660"},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"2481","DOI":"10.1109\/TPAMI.2016.2644615","article-title":"SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation","volume":"39","author":"Badrinarayanan","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_22","doi-asserted-by":"crossref","unstructured":"He, K., Zhang, X., Ren, S., and Sun, J. (2016, January 27\u201330). Deep residual learning for image recognition. Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA.","DOI":"10.1109\/CVPR.2016.90"},{"key":"ref_23","unstructured":"Simonyan, K., and Zisserman, A. (2015, January 7\u20139). Very deep convolutional networks for large-scale image recognition. Proceedings of the 3rd International Conference on Learning Representations ICLR 2015\u2014Conference Track Proceedings, San Diego, CA, USA."},{"key":"ref_24","doi-asserted-by":"crossref","unstructured":"Demir, I., Koperski, K., Lindenbaum, D., Pang, G., Huang, J., Basu, S., Hughes, F., Tuia, D., and Raskar, R. (2018, January 18\u201322). DeepGlobe 2018: A challenge to parse the earth through satellite images. Proceedings of the 2018 IEEE\/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), Salt Lake City, UT, USA.","DOI":"10.1109\/CVPRW.2018.00031"},{"key":"ref_25","unstructured":"Egenhofer, M.J., Sharma, J., and Mark, D.M. (November, January 30). A Critical Comparison of the 4-Intersection and 9-Intersection Models for Spatial Relations: Formal Analysis. Proceedings of the 11th Auto-Carto Conference, Minneapolis, MN, USA."},{"key":"ref_26","unstructured":"Randell, D.A., Cui, Z., and Cohn, A.G. (1992, January 25\u201329). A Spatial Logic based on Regions and Connection. Proceedings of the 3rd International Conference on Principles of Knowledge Representation and Reasoning, San Francisco, CA, USA."},{"key":"ref_27","doi-asserted-by":"crossref","first-page":"269","DOI":"10.1080\/02693799608902079","article-title":"Qualitative spatial reasoning: Cardinal directions as an example","volume":"10","author":"Frank","year":"1996","journal-title":"Int. J. Geogr. Inf. Syst."},{"key":"ref_28","doi-asserted-by":"crossref","unstructured":"Potnis, A.V., Durbha, S.S., and Kurte, K.R. (2018, January 22\u201327). A Geospatial Ontological Model for Remote Sensing Scene Semantic Knowledge Mining for the Flood Disaster. Proceedings of the IGARSS 2018\u20142018 IEEE International Geoscience and Remote Sensing Symposium, Valencia, Spain.","DOI":"10.1109\/IGARSS.2018.8517680"},{"key":"ref_29","doi-asserted-by":"crossref","first-page":"853","DOI":"10.1016\/0042-6989(94)00173-J","article-title":"Perceptual grouping by similarity and proximity: Experimental results can be predicted by intensity autocorrelations","volume":"35","author":"Sagi","year":"1995","journal-title":"Vis. Res."},{"key":"ref_30","doi-asserted-by":"crossref","first-page":"661","DOI":"10.3758\/BF03205537","article-title":"Uniform connectedness and classical gestalt principles of perceptual grouping","volume":"61","author":"Han","year":"1999","journal-title":"Percept. Psychophys."},{"key":"ref_31","unstructured":"Li, C., Parikh, D., and Chen, T. (2012, January 16\u201321). Automatic discovery of groups of objects for scene understanding. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Providence, RI, USA."},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"265","DOI":"10.1007\/s10339-017-0824-7","article-title":"Qualitative spatial logic descriptors from 3D indoor scenes to generate explanations in natural language","volume":"19","author":"Falomir","year":"2017","journal-title":"Cogn. Process."},{"key":"ref_33","doi-asserted-by":"crossref","unstructured":"Vinyals, O., Toshev, A., Bengio, S., and Erhan, D. (2015, January 7\u201312). Show and tell: A neural image caption generator. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7298935"},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"2891","DOI":"10.1109\/TPAMI.2012.162","article-title":"BabyTalk: Understanding and Generating Simple Image Descriptions","volume":"35","author":"Kulkarni","year":"2013","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"664","DOI":"10.1109\/TPAMI.2016.2598339","article-title":"Deep Visual-Semantic Alignments for Generating Image Descriptions","volume":"39","author":"Karpathy","year":"2017","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"57","DOI":"10.1017\/S1351324997001502","article-title":"Building applied natural language generation systems","volume":"3","author":"Reiter","year":"1997","journal-title":"Nat. Lang. Eng."},{"key":"ref_37","doi-asserted-by":"crossref","first-page":"907","DOI":"10.1006\/ijhc.1995.1081","article-title":"Toward principles for the design of ontologies used for knowledge sharing?","volume":"43","author":"Gruber","year":"1995","journal-title":"Int. J. Hum.-Comput. Stud."},{"key":"ref_38","doi-asserted-by":"crossref","unstructured":"Papineni, K., Roukos, S., Ward, T., and Zhu, W. (2002, January 7\u201312). BLEU: A Method for Automatic Evaluation of Machine Translation. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, PA, USA.","DOI":"10.3115\/1073083.1073135"},{"key":"ref_39","unstructured":"Banerjee, S., and Lavie, A. (2007, January 23). METEOR: An automatic metric for MT evaluation with improved correlation with human judgments. Proceedings of the Second Workshop on Statistical Machine Translation, Prague, Czech Republic."},{"key":"ref_40","unstructured":"Lin, C.Y. (2004, January 25\u201326). Rouge: A package for automatic evaluation of summaries. Proceedings of the Workshop on Text Summarization Branches Out (WAS 2004), Barcelona, Spain."},{"key":"ref_41","doi-asserted-by":"crossref","unstructured":"Vedantam, R., Zitnick, C.L., and Parikh, D. (2015, January 7\u201312). CIDEr: Consensus-based image description evaluation. Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA.","DOI":"10.1109\/CVPR.2015.7299087"},{"key":"ref_42","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1613\/jair.5714","article-title":"Learning Explanatory Rules from Noisy Data","volume":"61","author":"Evans","year":"2018","journal-title":"J. Artif. Intell. Res."}],"container-title":["ISPRS International Journal of Geo-Information"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2220-9964\/10\/1\/32\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T05:10:49Z","timestamp":1760159449000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2220-9964\/10\/1\/32"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,1,13]]},"references-count":42,"journal-issue":{"issue":"1","published-online":{"date-parts":[[2021,1]]}},"alternative-id":["ijgi10010032"],"URL":"https:\/\/doi.org\/10.3390\/ijgi10010032","relation":{},"ISSN":["2220-9964"],"issn-type":[{"type":"electronic","value":"2220-9964"}],"subject":[],"published":{"date-parts":[[2021,1,13]]}}}