{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,14]],"date-time":"2026-01-14T14:17:11Z","timestamp":1768400231085,"version":"3.49.0"},"reference-count":69,"publisher":"MDPI AG","issue":"6","license":[{"start":{"date-parts":[[2018,5,31]],"date-time":"2018-05-31T00:00:00Z","timestamp":1527724800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["J. Imaging"],"abstract":"<jats:p>Useful for human visual perception, edge detection remains a crucial stage in numerous image processing applications. One of the most challenging goals in contour detection is to operate algorithms that can process visual information as humans require. To ensure that an edge detection technique is reliable, it needs to be rigorously assessed before being used in a computer vision tool. This assessment corresponds to a supervised evaluation process to quantify differences between a reference edge map and a candidate, computed by a performance measure\/criterion. To achieve this task, a supervised evaluation computes a score between a ground truth edge map and a candidate image. This paper presents a survey of supervised edge detection evaluation methods. Considering a ground truth edge map, various methods have been developed to assess a desired contour. Several techniques are based on the number of false positive, false negative, true positive and\/or true negative points. Other methods strongly penalize misplaced points when they are outside a window centered on a true or false point. In addition, many approaches compute the distance from the position where a contour point should be located. Most of these edge detection assessment methods will be detailed, highlighting their drawbacks using several examples. In this study, a new supervised edge map quality measure is proposed. The new measure provides an overall evaluation of the quality of a contour map by taking into account the number of false positives and false negatives, and the degrees of shifting. Numerous examples and experiments show the importance of penalizing false negative points differently than false positive pixels because some false points may not necessarily disturb the visibility of desired objects, whereas false negative points can significantly change the aspect of an object. Finally, an objective assessment is performed by varying the hysteresis thresholds on contours of real images obtained by filtering techniques. Theoretically, by varying the hysteresis thresholds of the thin edges obtained by filtering gradient computations, the minimum score of the measure corresponds to the best edge map, compared to the ground truth. Twenty-eight measures are compared using different edge detectors that are robust or not robust regarding noise. The scores of the different measures and different edge detectors are recorded and plotted as a function of the noise level in the original image. The plotted curve of a reliable edge detection measure must increase monotonously with the noise level and a reliable edge detector must be less penalized than a poor detector. In addition, the obtained edge map tied to the minimum score of a considered measure exposes the reliability of an edge detection evaluation measure if the edge map obtained is visually closer to the ground truth or not. Hence, experiments illustrate that the desired objects are not always completely visible using ill-suited evaluation measure.<\/jats:p>","DOI":"10.3390\/jimaging4060074","type":"journal-article","created":{"date-parts":[[2018,6,1]],"date-time":"2018-06-01T03:02:50Z","timestamp":1527822170000},"page":"74","update-policy":"https:\/\/doi.org\/10.3390\/mdpi_crossmark_policy","source":"Crossref","is-referenced-by-count":33,"title":["A Review of Supervised Edge Detection Evaluation Methods and an Objective Comparison of Filtering Gradient Computations Using Hysteresis Thresholds"],"prefix":"10.3390","volume":"4","author":[{"given":"Baptiste","family":"Magnier","sequence":"first","affiliation":[{"name":"IMT Mines d\u2019Al\u00e8s, Laboratoire de G\u00e9nie Informatique et d\u2019Ing\u00e9nierie de Production (LGI2P), 6. avenue de Clavi\u00e8res, 30100 Al\u00e8s, France"}]},{"given":"Hasan","family":"Abdulrahman","sequence":"additional","affiliation":[{"name":"Northern Technical University, Department of Technical Computer systems, Kirkuk 36001, Iraki"}]},{"given":"Philippe","family":"Montesinos","sequence":"additional","affiliation":[{"name":"IMT Mines d\u2019Al\u00e8s, Laboratoire de G\u00e9nie Informatique et d\u2019Ing\u00e9nierie de Production (LGI2P), 6. avenue de Clavi\u00e8res, 30100 Al\u00e8s, France"}]}],"member":"1968","published-online":{"date-parts":[[2018,5,31]]},"reference":[{"key":"ref_1","first-page":"537","article-title":"Edge detection techniques: An overview","volume":"8","author":"Ziou","year":"1998","journal-title":"Int. J. on Patt. Rec. and Image Anal."},{"key":"ref_2","doi-asserted-by":"crossref","first-page":"898","DOI":"10.1109\/TPAMI.2010.161","article-title":"Contour detection and hierarchical image segmentation","volume":"33","author":"Arbelaez","year":"2011","journal-title":"IEEE TPAMI"},{"key":"ref_3","unstructured":"Sobel, I. (1970). Camera Models and Machine Perception. [Ph.D. Thesis, Stanford University]."},{"key":"ref_4","doi-asserted-by":"crossref","first-page":"679","DOI":"10.1109\/TPAMI.1986.4767851","article-title":"A computational approach to edge detection","volume":"6","author":"Canny","year":"1986","journal-title":"IEEE TPAMI"},{"key":"ref_5","first-page":"112","article-title":"An optimal linear operator for step edge detection","volume":"54","author":"Shen","year":"1992","journal-title":"CVGIP"},{"key":"ref_6","doi-asserted-by":"crossref","first-page":"167","DOI":"10.1007\/BF00123164","article-title":"Using Canny\u2019s criteria to derive a recursively implemented optimal edge detector","volume":"1","author":"Deriche","year":"1987","journal-title":"IJCV"},{"key":"ref_7","doi-asserted-by":"crossref","first-page":"1317","DOI":"10.1016\/S0165-1684(02)00283-9","article-title":"Generalization of Canny-Deriche filter for detection of noisy exponential edge","volume":"82","author":"Bourennane","year":"2002","journal-title":"Signal Proces."},{"key":"ref_8","first-page":"187","article-title":"Theory of edge detection","volume":"207","author":"Marr","year":"1980","journal-title":"Proc. R. Soc. Lond. Ser. B Biol. Sci."},{"key":"ref_9","doi-asserted-by":"crossref","first-page":"891","DOI":"10.1109\/34.93808","article-title":"The Design and Use of Steerable Filters","volume":"13","author":"Freeman","year":"1991","journal-title":"IEEE TPAMI"},{"key":"ref_10","doi-asserted-by":"crossref","first-page":"1007","DOI":"10.1109\/TPAMI.2004.44","article-title":"Design of steerable filters for feature detection using Canny-like criteria","volume":"26","author":"Jacob","year":"2004","journal-title":"IEEE TPAMI"},{"key":"ref_11","doi-asserted-by":"crossref","unstructured":"Geusebroek, J., Smeulders, A., and van de Weijer, J. (2002). Fast anisotropic Gauss filtering. European Conference on Computer Vision, Springer.","DOI":"10.1007\/3-540-47969-4_7"},{"key":"ref_12","unstructured":"Magnier, B., Montesinos, P., and Diep, D. (2011, January 4\u20136). Fast anisotropic edge detection using Gamma correction in color images. Proceedings of the 7th International Symposium on Image and Signal Processing and Analysis (ISPA 2011), Dubrovnik, Croatia."},{"key":"ref_13","unstructured":"Martin, D., Fowlkes, C., Tal, D., and Malik, J. (2001, January 7\u201314). A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. Proceedings of the Eighth IEEE International Conference on Computer Vision, Vancouver, BC, Canada."},{"key":"ref_14","first-page":"133","article-title":"From contours to ground truth: How to evaluate edge detectors by filtering","volume":"25","author":"Abdulrahman","year":"2017","journal-title":"J. WSCG"},{"key":"ref_15","doi-asserted-by":"crossref","first-page":"1338","DOI":"10.1109\/34.643893","article-title":"A robust visual method for assessing the relative performance of edge-detection algorithms","volume":"19","author":"Heath","year":"1997","journal-title":"IEEE TPAMI"},{"key":"ref_16","doi-asserted-by":"crossref","first-page":"597","DOI":"10.1109\/TSMC.1981.4308758","article-title":"Edge evaluation using local edge coherence","volume":"11","author":"Kitchen","year":"1981","journal-title":"IEEE Trans. Syst. Man Cybern."},{"key":"ref_17","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/0031-3203(90)90045-M","article-title":"Context dependent edge detection and evaluation","volume":"23","author":"Haralick","year":"1990","journal-title":"Pattern Recognit."},{"key":"ref_18","doi-asserted-by":"crossref","first-page":"21","DOI":"10.1016\/0262-8856(95)01036-X","article-title":"Efficient evaluations of edge connectivity and width uniformity","volume":"14","author":"Zhu","year":"1996","journal-title":"Image Vis. Comput."},{"key":"ref_19","doi-asserted-by":"crossref","first-page":"205","DOI":"10.1109\/TC.1978.1675073","article-title":"A quantitative study of the orientation bias of some edge detector schemes","volume":"3","author":"Deutsch","year":"1978","journal-title":"IEEE Trans. Comput."},{"key":"ref_20","doi-asserted-by":"crossref","first-page":"23","DOI":"10.1016\/1049-9652(92)90031-R","article-title":"Edge evaluation using necessary components","volume":"54","author":"Venkatesh","year":"1992","journal-title":"CVGIP Graph. Models Image Process."},{"key":"ref_21","first-page":"1","article-title":"Edge detection: A review of dissimilarity evaluations and a proposed normalized measure","volume":"77","author":"Magnier","year":"2017","journal-title":"Multimed. Tools Appl."},{"key":"ref_22","doi-asserted-by":"crossref","first-page":"944","DOI":"10.1117\/12.130263","article-title":"An adaptable edge quality metric","volume":"32","author":"Strickland","year":"1993","journal-title":"Optical Eng."},{"key":"ref_23","doi-asserted-by":"crossref","first-page":"805","DOI":"10.1016\/S0167-8655(00)00045-3","article-title":"Contextual and non-contextual performance evaluation of edge detectors","volume":"21","author":"Nguyen","year":"2000","journal-title":"Pattern Recognit. Lett."},{"key":"ref_24","first-page":"566","article-title":"A modified Hausdorff distance for object matching","volume":"1","author":"Dubuisson","year":"1994","journal-title":"IEEE ICPR"},{"key":"ref_25","doi-asserted-by":"crossref","first-page":"2","DOI":"10.1155\/2008\/693053","article-title":"Comparative study of contour detection evaluation criteria based on dissimilarity measures","volume":"2008","author":"Chabrier","year":"2008","journal-title":"EURASIP J. Image Video Process."},{"key":"ref_26","doi-asserted-by":"crossref","first-page":"1125","DOI":"10.1016\/j.patcog.2012.10.027","article-title":"Quantitative error measures for edge detection","volume":"46","author":"Bustince","year":"2013","journal-title":"Pattern Recognit."},{"key":"ref_27","first-page":"223","article-title":"Nouvelles recherches sur la distribution florale","volume":"44","author":"Jaccard","year":"1908","journal-title":"Bulletin de la Societe Vaudoise des Sciences Naturelles"},{"key":"ref_28","doi-asserted-by":"crossref","first-page":"297","DOI":"10.2307\/1932409","article-title":"Measures of the amount of ecologic association between species","volume":"26","author":"Dice","year":"1945","journal-title":"Ecology"},{"key":"ref_29","unstructured":"Sneath, P., and Sokal, R. (1973). Numerical Taxonomy. The Principles and Practice of Numerical Classification, The University of Chicago Press."},{"key":"ref_30","unstructured":"Duda, R., Hart, P., and Stork, D. (1995). Pattern Classification and Scene Analysis, Wiley Interscience. [2nd ed.]."},{"key":"ref_31","first-page":"729","article-title":"Contour detection based on nonclassical receptive field inhibition","volume":"12","author":"Grigorescu","year":"2003","journal-title":"IEEE TIP"},{"key":"ref_32","doi-asserted-by":"crossref","first-page":"076278","DOI":"10.1155\/ASP\/2006\/76278","article-title":"Evaluating edge detection through boundary detection","volume":"2006","author":"Wang","year":"2006","journal-title":"EURASIP J. Appl. Signal Process."},{"key":"ref_33","unstructured":"Bryant, D., and Bouldin, D. (1979, January 6\u20138). Evaluation of edge operators using relative and absolute grading. Proceedings of the Conference on Pattern Recognition and Image Processing, Chicago, IL, USA."},{"key":"ref_34","doi-asserted-by":"crossref","first-page":"021746","DOI":"10.1155\/ASP\/2006\/21746","article-title":"A method for assessment of segmentation success considering uncertainty in the edge positions","volume":"2006","author":"Usamentiaga","year":"2006","journal-title":"EURASIP J. Appl. Signal Proc."},{"key":"ref_35","first-page":"171","article-title":"A comparative performance study of several global thresholding techniques for segmentation","volume":"52","author":"Lee","year":"1990","journal-title":"CVGIP"},{"key":"ref_36","doi-asserted-by":"crossref","first-page":"146","DOI":"10.1117\/1.1631315","article-title":"Survey over image thresholding techniques and quantitative performance evaluation","volume":"13","author":"Sezgin","year":"2004","journal-title":"J. Electron. Imaging"},{"key":"ref_37","first-page":"146","article-title":"Dynamic threshold determination by local and global edge evaluation","volume":"57","author":"Venkatesh","year":"1995","journal-title":"CVGIP"},{"key":"ref_38","doi-asserted-by":"crossref","first-page":"1027","DOI":"10.1109\/TPAMI.2003.1217608","article-title":"A method for objective edge detection evaluation and detector parameter selection","volume":"25","author":"Yitzhaky","year":"2003","journal-title":"IEEE TPAMI"},{"key":"ref_39","doi-asserted-by":"crossref","first-page":"530","DOI":"10.1109\/TPAMI.2004.1273918","article-title":"Learning to detect natural image boundaries using local brightness, color, and texture cues","volume":"26","author":"Martin","year":"2004","journal-title":"IEEE TPAMI"},{"key":"ref_40","doi-asserted-by":"crossref","first-page":"77","DOI":"10.1006\/cviu.2001.0931","article-title":"Edge detector evaluation using empirical ROC curves","volume":"84","author":"Bowyer","year":"2001","journal-title":"Comput. Vis. Image Underst."},{"key":"ref_41","unstructured":"Forbes, L.A., and Draper, B.A. (2000, January 15). Inconsistencies in edge detector evaluation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Hilton Head Island, SC, USA."},{"key":"ref_42","doi-asserted-by":"crossref","unstructured":"Davis, J., and Goadrich, M. (2006, January 25\u201329). The relationship between Precision\u2013Recall and ROC curves. Proceedings of the 23rd International Conference on Machine Learning, Pittsburgh, PA, USA.","DOI":"10.1145\/1143844.1143874"},{"key":"ref_43","doi-asserted-by":"crossref","unstructured":"Hou, X., Yuille, A., and Koch, C. (2013). Boundary detection benchmarking: Beyond F-measures. 2013 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), IEEE.","DOI":"10.1109\/CVPR.2013.276"},{"key":"ref_44","unstructured":"Valverde, F.L., Guil, N., Munoz, J., Nishikawa, R., and Doi, K. (2001, January 7\u201310). An evaluation criterion for edge detection techniques in noisy images. Proceedings of the International Conference on Image Processing, Thessaloniki, Greece."},{"key":"ref_45","doi-asserted-by":"crossref","first-page":"969","DOI":"10.1016\/S0031-3203(00)00052-2","article-title":"A measure of quality for evaluating methods of segmentation and edge detection","volume":"34","year":"2001","journal-title":"Pattern Recognit."},{"key":"ref_46","doi-asserted-by":"crossref","first-page":"35","DOI":"10.1016\/j.patrec.2003.08.011","article-title":"Characterization of empirical discrepancy evaluation measures","volume":"25","year":"2004","journal-title":"Pattern Recognit. Lett."},{"key":"ref_47","doi-asserted-by":"crossref","first-page":"753","DOI":"10.1109\/PROC.1979.11325","article-title":"Quantitative design and evaluation of enhancement\/thresholding edge detectors","volume":"67","author":"Abdou","year":"1979","journal-title":"Proc. IEEE"},{"key":"ref_48","doi-asserted-by":"crossref","unstructured":"Pinho, A.J., and Almeida, L.B. (1995). Edge detection filters based on artificial neural networks. ICIAP, Springer.","DOI":"10.1007\/3-540-60298-4_252"},{"key":"ref_49","unstructured":"Boaventura, A.G., and Gonzaga, A. (2006). Method to evaluate the performance of edge detector. Brazlian Symp. on Comput. Graph. Image Process, Citeseer."},{"key":"ref_50","doi-asserted-by":"crossref","first-page":"1505","DOI":"10.1109\/TSMC.2015.2503386","article-title":"A New Reference-Based Edge Map Quality Measure","volume":"46","author":"Panetta","year":"2016","journal-title":"IEEE Trans. Syst. Man Cybern. Syst."},{"key":"ref_51","first-page":"107","article-title":"Error measures for objective assessment of scene segmentation algorithms","volume":"1","author":"Yasnoff","year":"1978","journal-title":"Anal. Quant. Cytol."},{"key":"ref_52","unstructured":"Huttenlocher, D., and Rucklidge, W. (1993, January 15\u201317). A multi-resolution technique for comparing images using the hausdorff distance. Proceedings of the Computer Vision and Pattern Recognition (IEEE CVPR), New York, NY, USA."},{"key":"ref_53","doi-asserted-by":"crossref","unstructured":"Peli, T., and Malah, D. (1982). A Study of Edge Detection Algorithms, CGIP.","DOI":"10.1016\/0146-664X(82)90070-3"},{"key":"ref_54","unstructured":"Odet, C., Belaroussi, B., and Benoit-Cattin, H. (2002, January 22\u201325). Scalable discrepancy measures for segmentation evaluation. Proceedings of the 2002 International Conference on Image Processing, Rochester, NY, USA."},{"key":"ref_55","doi-asserted-by":"crossref","first-page":"353","DOI":"10.1109\/TSMCB.2007.912940","article-title":"Edge enhancement nucleus and cytoplast contour detector of cervical smear images","volume":"38","author":"Chan","year":"2008","journal-title":"IEEE Trans. Syst. Man Cybern. Part B"},{"key":"ref_56","doi-asserted-by":"crossref","unstructured":"Magnier, B. (2018, January 2\u20134). An objective evaluation of edge detection methods based on oriented half kernels. Proceedings of the Illinois Consortium for International Studies and Programs (ICISP), Normandy, France.","DOI":"10.1007\/978-3-319-94211-7_10"},{"key":"ref_57","unstructured":"Baddeley, A.J. (1992). An error metric for binary images. Robust Computer Vision: Quality of Vision Algorithms, Wichmann."},{"key":"ref_58","doi-asserted-by":"crossref","unstructured":"Magnier, B., Le, A., and Zogo, A. (2016, January 4\u20136). A Quantitative Error Measure for the Evaluation of Roof Edge Detectors. Proceedings of the 2016 IEEE International Conference on Imaging Systems and Techniques (IST), Chania, Greece.","DOI":"10.1109\/IST.2016.7738264"},{"key":"ref_59","doi-asserted-by":"crossref","unstructured":"Abdulrahman, H., Magnier, B., and Montesinos, P. (2017). A New Objective Supervised Edge Detection Assessment using Hysteresis Thresholds. International Conference on Image Analysis and Processing, Springer.","DOI":"10.1007\/978-3-319-70742-6_1"},{"key":"ref_60","unstructured":"Chabrier, S., Laurent, H., Emile, B., Rosenberger, C., and Marche, P. (2004, January 6\u201310). A comparative study of supervised evaluation criteria for image segmentation. Proceedings of the European Signal Processing Conference, Vienna, Austria."},{"key":"ref_61","doi-asserted-by":"crossref","first-page":"023017","DOI":"10.1117\/1.3446803","article-title":"Comparative study of localization metrics for the evaluation of image interpretation systems","volume":"19","author":"Hemery","year":"2010","journal-title":"J. Electron. Imaging"},{"key":"ref_62","doi-asserted-by":"crossref","first-page":"1057","DOI":"10.1016\/S0167-8655(97)80002-5","article-title":"Robust comparison of binary images","volume":"18","author":"Paumard","year":"1997","journal-title":"Pattern Recognit. Lett."},{"key":"ref_63","doi-asserted-by":"crossref","first-page":"581","DOI":"10.1016\/j.patrec.2004.09.022","article-title":"A new Hausdorff distance for image matching","volume":"26","author":"Zhao","year":"2005","journal-title":"Pattern Recognit. Lett."},{"key":"ref_64","doi-asserted-by":"crossref","first-page":"1461","DOI":"10.1016\/j.patcog.2007.07.011","article-title":"Binary-image comparison with local-dissimilarity quantification","volume":"41","author":"Baudrier","year":"2008","journal-title":"Pattern Recognit."},{"key":"ref_65","first-page":"3","article-title":"Steerable-scalable kernels for edge detection and junction analysis","volume":"Volume 10","author":"Perona","year":"1992","journal-title":"European Conference on Computer Vision"},{"key":"ref_66","doi-asserted-by":"crossref","first-page":"806","DOI":"10.1016\/j.patcog.2011.07.020","article-title":"Noise-robust edge detector combining isotropic and anisotropic Gaussian kernels","volume":"45","author":"Shui","year":"2012","journal-title":"Pattern Recognit."},{"key":"ref_67","doi-asserted-by":"crossref","first-page":"185","DOI":"10.1109\/LSP.2006.884030","article-title":"Regularization preserving localization of close edges","volume":"14","author":"Laligant","year":"2007","journal-title":"IEEE Signal Process. Lett."},{"key":"ref_68","doi-asserted-by":"crossref","first-page":"1106","DOI":"10.1109\/34.42841","article-title":"Localization and noise in edge detection","volume":"11","author":"Caprile","year":"1989","journal-title":"IEEE Trans. Pattern Anal. Mach. Intell."},{"key":"ref_69","doi-asserted-by":"crossref","first-page":"1047","DOI":"10.1109\/TIP.2015.2510284","article-title":"Separability criteria for the evaluation of boundary detection benchmarks","volume":"25","author":"Bustince","year":"2016","journal-title":"IEEE Trans. Image Process."}],"container-title":["Journal of Imaging"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.mdpi.com\/2313-433X\/4\/6\/74\/pdf","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,10,11]],"date-time":"2025-10-11T15:06:44Z","timestamp":1760195204000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.mdpi.com\/2313-433X\/4\/6\/74"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2018,5,31]]},"references-count":69,"journal-issue":{"issue":"6","published-online":{"date-parts":[[2018,6]]}},"alternative-id":["jimaging4060074"],"URL":"https:\/\/doi.org\/10.3390\/jimaging4060074","relation":{},"ISSN":["2313-433X"],"issn-type":[{"value":"2313-433X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2018,5,31]]}}}