{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,6]],"date-time":"2026-06-06T15:25:37Z","timestamp":1780759537387,"version":"3.54.1"},"reference-count":58,"publisher":"MDPI AG","issue":"18","license":[{"start":{"date-parts":[[2019,9,12]],"date-time":"2019-09-12T00:00:00Z","timestamp":1568246400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Sensors"],"abstract":"<jats:p>Prediction of possible landslide areas is the first stage of landslide hazard mitigation efforts and is also crucial for suitable site selection. Several statistical and machine learning methodologies have been applied for the production of landslide susceptibility maps. However, the performance assessment of such methods have conventionally been carried out by utilizing existing landslide inventories. The purpose of this study is to investigate the performances of landslide susceptibility maps produced with three different machine learning algorithms, i.e., random forest, artificial neural network, and logistic regression, in a recently constructed and activated dam reservoir and assess the external quality of each map by using pre- and post-event photogrammetric datasets. The methodology introduced here was applied using digital surface models generated from aerial photogrammetric flight data acquired before and after the dam construction. Aerial photogrammetric images acquired in 2012 and 2018 (after the dam was filled) were used to produce digital terrain models and orthophotos. The 2012 dataset was used for producing the landslide susceptibility maps and the results were evaluated by comparing the Euclidian distances between the two surface models. The results show that the random forest method outperforms the other two for predicting the future landslides.<\/jats:p>","DOI":"10.3390\/s19183940","type":"journal-article","created":{"date-parts":[[2019,9,12]],"date-time":"2019-09-12T10:56:06Z","timestamp":1568285766000},"page":"3940","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":163,"title":["A Novel Performance Assessment Approach Using Photogrammetric Techniques for Landslide Susceptibility Mapping with Logistic Regression, ANN and Random Forest"],"prefix":"10.3390","volume":"19","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-1682-4923","authenticated-orcid":false,"given":"Eray","family":"Sevgen","sequence":"first","affiliation":[{"name":"Department of Geomatics Engineering, Hacettepe University, 06800 Beytepe Ankara, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-2775-7914","authenticated-orcid":false,"given":"Sultan","family":"Kocaman","sequence":"additional","affiliation":[{"name":"Department of Geomatics Engineering, Hacettepe University, 06800 Beytepe Ankara, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1117-6012","authenticated-orcid":false,"given":"Hakan A.","family":"Nefeslioglu","sequence":"additional","affiliation":[{"name":"Department of Geological Engineering, Hacettepe University, 06800 Beytepe Ankara, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4762-9933","authenticated-orcid":false,"given":"Candan","family":"Gokceoglu","sequence":"additional","affiliation":[{"name":"Department of Geological Engineering, Hacettepe University, 06800 Beytepe Ankara, Turkey"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"1968","published-online":{"date-parts":[[2019,9,12]]},"reference":[{"key":"ref_1","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1016\/j.cageo.2011.04.012","article-title":"An easy-to-use MATLAB program (MamLand) for the assessment of landslide susceptibility using a Mamdani fuzzy algorithm","volume":"38","author":"Akgun","year":"2012","journal-title":"Comput. Geosci."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"20","DOI":"10.1016\/j.cageo.2013.12.016","article-title":"GeoFIS: An integrated tool for the assessment of landslide susceptibility","volume":"66","author":"Osna","year":"2014","journal-title":"Comput. Geosci."},{"key":"ref_3","doi-asserted-by":"crossref","first-page":"26","DOI":"10.1016\/j.cageo.2016.10.001","article-title":"An expert-based landslide susceptibility mapping (LSM) module developed for Netcad Architect Software","volume":"98","author":"Sezer","year":"2017","journal-title":"Comput. Geosci."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"67","DOI":"10.1016\/j.enggeo.2015.12.020","article-title":"Engineering geology\u2014A fifty year perspective","volume":"201","author":"Juang","year":"2016","journal-title":"Eng. Geol."},{"key":"ref_5","doi-asserted-by":"crossref","first-page":"345","DOI":"10.1007\/s10346-009-0166-3","article-title":"A statistical assessment on international landslide literature (1945\u20132008)","volume":"6","author":"Gokceoglu","year":"2009","journal-title":"Landslides"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"60","DOI":"10.1016\/j.earscirev.2018.03.001","article-title":"A review of statistically-based landslide susceptibility models","volume":"180","author":"Reichenbach","year":"2018","journal-title":"Earth Sci. Rev."},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"171","DOI":"10.1016\/j.enggeo.2008.01.004","article-title":"An assessment on the use of logistic regression and artificial neural networks with different sampling strategies for the preparation of landslide susceptibility maps","volume":"97","author":"Nefeslioglu","year":"2008","journal-title":"Eng. Geol."},{"key":"ref_8","doi-asserted-by":"crossref","first-page":"27","DOI":"10.1007\/s10064-014-0607-7","article-title":"A comparative study of landslide susceptibility mapping using landslide susceptibility index and artificial neural networks in the Krios River and Krathis River catchments (northern Peloponnesus, Greece)","volume":"74","author":"Polykretis","year":"2015","journal-title":"Bull. Eng. Geol. Environ."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"361","DOI":"10.1007\/s10346-015-0557-6","article-title":"Spatial prediction models for shallow landslide hazards: A comparative assessment of the efficacy of support vector machines, artificial neural networks, kernel logistic regression, and logistic model tree","volume":"13","author":"Tuan","year":"2016","journal-title":"Landslides"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"367","DOI":"10.1080\/10106049.2016.1140824","article-title":"A GIS-based comparative study of Dempster-Shafer, logistic regression and artificial neural network models for landslide susceptibility mapping","volume":"32","author":"Chen","year":"2017","journal-title":"Geocarto Int."},{"key":"ref_11","doi-asserted-by":"crossref","first-page":"157","DOI":"10.1016\/j.ijsrc.2017.09.008","article-title":"A hybrid machine learning ensemble approach based on a radial basis function neural network and rotation forest for landslide susceptibility modeling: A case study in the Himalayan area, India","volume":"33","author":"Pham","year":"2018","journal-title":"Int. J. Sediment Res."},{"key":"ref_12","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1080\/19475705.2018.1487471","article-title":"Mapping earthquake-triggered landslide susceptibility by use of artificial neural network (ANN) models an example of the 2013 Minxian (China) Mw 5.9 event","volume":"10","author":"Tian","year":"2019","journal-title":"Geomat. Nat. Hazards Risk"},{"key":"ref_13","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1016\/j.geomorph.2004.06.010","article-title":"The application of GIS-based logistic regression for landslide susceptibility mapping in the Kakuda Yahiko Mountains, Central Japan","volume":"65","author":"Ayalew","year":"2005","journal-title":"Geomorphology"},{"key":"ref_14","doi-asserted-by":"crossref","first-page":"250","DOI":"10.1016\/j.geomorph.2005.05.011","article-title":"Susceptibility assessments of shallow earthflows triggered by heavy rainfall at three catchments by logistic regression analyses","volume":"72","author":"Can","year":"2005","journal-title":"Geomorphology"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"241","DOI":"10.1007\/s00254-006-0322-1","article-title":"Application of logistic regression for landslide susceptibility zoning of Cekmece Area, Istanbul, Turkey","volume":"51","author":"Duman","year":"2006","journal-title":"Environ. Geol."},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"323","DOI":"10.1007\/s11069-007-9190-6","article-title":"Implementation of reconstructed geomorphologic units in landslide susceptibility mapping: The Melen Gorge (NW Turkey)","volume":"46","author":"Gorum","year":"2008","journal-title":"Nat. Hazards"},{"key":"ref_17","first-page":"40","article-title":"An assessment on the use of Terra ASTER L3A data in landslide susceptibility mapping","volume":"14","author":"Nefeslioglu","year":"2012","journal-title":"J. Appl. Earth Obs. Geoinf."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"565","DOI":"10.1007\/s11004-014-9560-z","article-title":"Spatial landslide hazard prediction using rainfall probability and a logistic regression model","volume":"47","author":"Lee","year":"2015","journal-title":"Math. Geosci."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"164","DOI":"10.1016\/j.catena.2016.06.004","article-title":"Comparison of a logistic regression and Naive Bayes classifier in landslide susceptibility assessments: The influence of models complexity and training dataset size","volume":"145","author":"Tsangaratos","year":"2016","journal-title":"Catena"},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"249","DOI":"10.1007\/s11629-016-4126-9","article-title":"Landslide susceptibility mapping using integrated model of information value method and logistic regression in the Bailongjiang watershed, Gansu province, China","volume":"14","author":"Du","year":"2017","journal-title":"J. Mt. Sci."},{"key":"ref_21","doi-asserted-by":"crossref","first-page":"105","DOI":"10.1016\/j.geomorph.2016.02.012","article-title":"Landslide susceptibility assessment in Lianhua County (China): A comparison between random forest data mining technique and bivariate and multivariate statistical models","volume":"259","author":"Hong","year":"2016","journal-title":"Geomorphology"},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"45","DOI":"10.1016\/j.catena.2018.12.035","article-title":"Exploring the effects of the design and quantity of absence data on the performance of random forest-based landslide susceptibility mapping","volume":"176","author":"Hong","year":"2019","journal-title":"Catena"},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"147","DOI":"10.1016\/j.catena.2016.11.032","article-title":"A comparative study of logistic model tree, random forest, and classification and regression tree models for spatial prediction of landslide susceptibility","volume":"151","author":"Chen","year":"2017","journal-title":"Catena"},{"key":"ref_24","doi-asserted-by":"crossref","first-page":"341","DOI":"10.1007\/s12303-018-0038-8","article-title":"Comparison of landslide susceptibility maps using random forest and multivariate adaptive regression spline models in combination with catchment map units","volume":"23","author":"Chu","year":"2019","journal-title":"Geosci. J."},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"839","DOI":"10.1007\/s10346-015-0614-1","article-title":"Landslide susceptibility mapping using random forest, boosted regression tree, classification and regression tree, and general linear models and comparison of their performance at Wadi Tayyah Basin, Asir Region, Saudi Arabia","volume":"13","author":"Youssef","year":"2016","journal-title":"Landslides"},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"332","DOI":"10.1016\/j.scitotenv.2019.01.221","article-title":"Assessment of advanced random forest and decision tree algorithms for modeling rainfall-induced landslide susceptibility in the Izu-Oshima Volcanic Island, Japan","volume":"662","author":"Dou","year":"2019","journal-title":"Sci. Total Environ."},{"key":"ref_27","unstructured":"Akba\u015f, B., Akdeniz, N., Aksay, A., Altun, \u0130., Balc\u0131, V., Bilginer, E., Bilgi\u00e7, T., Duru, M., Ercan, T., and Gedik, \u0130. (2016). T\u00fcrkiye Jeoloji Haritas\u0131."},{"key":"ref_28","first-page":"133","article-title":"Do\u011fu Anadolu\u2019da Solhan (Mu\u015f) Volkanitleri\u2019nin Petrojenetik \u0130ncelenmesi","volume":"14","author":"Yilmaz","year":"1987","journal-title":"Yerbilimleri"},{"key":"ref_29","unstructured":"Emre, \u00d6., Duman, T.Y., \u00d6zalp, S., Elmac\u0131, H., Olgun, \u015e., and \u015earo\u011flu, F. (2013). 1\/1.250.000 \u00d6l\u00e7ekli T\u00fcrkiye Diri Fay Haritas\u0131, Maden Tetkik ve Arama Genel M\u00fcd\u00fcrl\u00fc\u011f\u00fc \u00d6zel Yay\u0131nlar Serisi."},{"key":"ref_30","unstructured":"MGM (2016). K\u00f6ppen \u0130klim S\u0131n\u0131fland\u0131rmas\u0131na G\u00f6re Turkiye \u0130klimi, Ara\u015ft\u0131rma Dairesi Ba\u015fkanl\u0131\u011f\u0131, Klimatoloji \u015eube M\u00fcd\u00fcrl\u00fc\u011f\u00fc."},{"key":"ref_31","unstructured":"(2019, August 13). MGM: Meteoroloji Genel M\u00fcd\u00fcrl\u00fc\u011f\u00fc, Available online: https:\/\/www.mgm.gov.tr\/."},{"key":"ref_32","unstructured":"Turner, A.K., and Schuster, R.L. (1996). Landslide types and processes. Landslides: Investigation and Mitigation, Transportation Research Board. National Research Council Transportation Research Board Special Report (Book 247)."},{"key":"ref_33","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1038\/ngeo776","article-title":"Landslide erosion controlled by hillslope material","volume":"3","author":"Larsen","year":"2010","journal-title":"Nat. Geosci."},{"key":"ref_34","doi-asserted-by":"crossref","unstructured":"IAEG (Commision on Landslides) (1990). Suggested nomenclature for landslides. Bull. Int. Assoc. Eng. Geol., 41, 13\u201316.","DOI":"10.1007\/BF02590202"},{"key":"ref_35","doi-asserted-by":"crossref","first-page":"151","DOI":"10.1016\/j.isprsjprs.2005.02.006","article-title":"Least squares 3D surface and curve matching","volume":"59","author":"Gruen","year":"2005","journal-title":"ISPRS J. Photogramm. Remote Sens."},{"key":"ref_36","unstructured":"(2019, June 02). HGM: Sayisal Uretim Faaliyetlerinde Kilometre Taslari, Available online: https:\/\/www.harita.gov.tr\/kilometre-taslari."},{"key":"ref_37","unstructured":"Wilson, J.P., and Gallant, J.C. (2000). Terrain Analysis Principles and Applications, John Wiley and Sons, Inc."},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"3","DOI":"10.1002\/hyp.3360050103","article-title":"Digital terrain modeling: A review of hydrological, geomorphological, and biological applications","volume":"5","author":"Moore","year":"1991","journal-title":"Hydrol. Process."},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"65","DOI":"10.1016\/j.enggeo.2005.07.011","article-title":"The 17 March 2005 Kuzulu landslide (Sivas, Turkey) and landslide-susceptibility map of its near vicinity","volume":"81","author":"Gokceoglu","year":"2005","journal-title":"Eng. Geol."},{"key":"ref_40","doi-asserted-by":"crossref","unstructured":"Hosmer, D.W., and Lemeshow, S. (2000). Applied Logistic Regression, John Wiley & Sons.","DOI":"10.1002\/0471722146"},{"key":"ref_41","unstructured":"Kleinbaum, D.G., and Klein, M. (2002). Logistic Regression: A Self Learning Text, Springer. [2nd ed.]."},{"key":"ref_42","unstructured":"Negnevitsky, M. (2002). Artificial Intelligence\u2014A Guide to Intelligent Systems, Addison-Wesley Co."},{"key":"ref_43","doi-asserted-by":"crossref","first-page":"5","DOI":"10.1023\/A:1010933404324","article-title":"Random Forests","volume":"45","author":"Breiman","year":"2001","journal-title":"Mach. Learn."},{"key":"ref_44","first-page":"2825","article-title":"Scikit-learn: Machine Learning in Python","volume":"12","author":"Pedregosa","year":"2011","journal-title":"J. Mach. Learn. Res."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"1145","DOI":"10.1016\/S0031-3203(96)00142-2","article-title":"The use of the area under the ROC curve in the evaluation of machine learning algorithms","volume":"30","author":"Bradley","year":"1997","journal-title":"Pattern Recognit."},{"key":"ref_46","unstructured":"Zhang, L., Kocaman, S., Akca, D., Kornus, W., and Baltsavias, E. (2006, January 3\u20136). Test and performance evaluation of DMC images and new methods for their processing. Proceedings of the ISPRS Commission I Symposium, Paris, France."},{"key":"ref_47","first-page":"14","article-title":"3D modeling of the Weary Herakles statue with a coded structured light system","volume":"Volume XXXVI","author":"Akca","year":"2006","journal-title":"International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, Proceedings of the ISPRS Commission V Symposium, Dresden, Germany, 25\u201327 September 2006"},{"key":"ref_48","doi-asserted-by":"crossref","first-page":"L08402","DOI":"10.1029\/2009GL041605","article-title":"Topography of inland deltas: Observations, modeling, and experiments","volume":"37","author":"Seybold","year":"2010","journal-title":"Geophys. Res. Lett."},{"key":"ref_49","unstructured":"Akca, D. (2007). Least Squares 3D Surface Matching. [Ph.D. Thesis, Institute of Geodesy and Photogrammetry]."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"307","DOI":"10.14358\/PERS.76.3.307","article-title":"Co-registration of surfaces by 3D Least Squares matching","volume":"76","author":"Akca","year":"2010","journal-title":"Photogramm. Eng. Remote Sens."},{"key":"ref_51","doi-asserted-by":"crossref","first-page":"121","DOI":"10.1111\/phor.12149","article-title":"Monitoring of a laboratory-scale inland-delta formation using a structured-light system","volume":"31","author":"Akca","year":"2016","journal-title":"Photogramm. Rec."},{"key":"ref_52","doi-asserted-by":"crossref","first-page":"1285","DOI":"10.1126\/science.3287615","article-title":"Measuring the accuracy of diagnostic systems","volume":"240","author":"Swets","year":"1998","journal-title":"Science"},{"key":"ref_53","doi-asserted-by":"crossref","first-page":"173","DOI":"10.1007\/s11069-018-3536-0","article-title":"Landslide susceptibility assessment at the Wuning area, China: A comparison between multi-criteria decision making, bivariate statistical and machine learning methods","volume":"96","author":"Hong","year":"2019","journal-title":"Nat. Hazards"},{"key":"ref_54","doi-asserted-by":"crossref","first-page":"247","DOI":"10.1007\/s10064-018-1256-z","article-title":"Spatial prediction of landslide susceptibility using data mining-based kernel logistic regression, naive Bayes and RBF Network models for the Long County area (China)","volume":"78","author":"Chen","year":"2019","journal-title":"Bull. Eng. Geol. Environ."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"43","DOI":"10.1080\/10106049.2017.1357767","article-title":"Improving the reliability of landslide susceptibility mapping through spatial uncertainty analysis: A case study of Al Hoceima, Northern Morocco","volume":"34","author":"Rahali","year":"2019","journal-title":"Geocarto Int."},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Ozer, B.C., Mutlu, B., Nefeslioglu, H.A., Sezer, E.A., Rouai, M., Dekayir, A., and Gokceoglu, C. (2019). On the use of hierarchical fuzzy inference systems (HFIS) in expert-based landslide susceptibility mapping: The central part of the Rif Mountains (Morocco). Bull. Eng. Geol. Environ.","DOI":"10.1007\/s10064-019-01548-5"},{"key":"ref_57","doi-asserted-by":"crossref","first-page":"611","DOI":"10.1007\/s10346-018-1101-2","article-title":"A CitSci app for landslide data collection","volume":"16","author":"Kocaman","year":"2019","journal-title":"Landslides"},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Can, R., Kocaman, S., and Gokceoglu, C. (2019). A Convolutional neural network architecture for auto-detection of landslide photographs to assess citizen science and volunteered geographic information data quality. ISPRS Int. J. Geo Inf., 8.","DOI":"10.3390\/ijgi8070300"}],"container-title":["Sensors"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/18\/3940\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T13:19:24Z","timestamp":1760188764000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/1424-8220\/19\/18\/3940"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2019,9,12]]},"references-count":58,"journal-issue":{"issue":"18","published-online":{"date-parts":[[2019,9]]}},"alternative-id":["s19183940"],"URL":"https:\/\/doi.org\/10.3390\/s19183940","relation":{},"ISSN":["1424-8220"],"issn-type":[{"value":"1424-8220","type":"electronic"}],"subject":[],"published":{"date-parts":[[2019,9,12]]}}}