{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,10,17]],"date-time":"2026-10-17T08:28:06Z","timestamp":1792225686328,"version":"4.4.0"},"reference-count":60,"publisher":"Association for Computing Machinery (ACM)","issue":"3","license":[{"start":{"date-parts":[[2023,10,6]],"date-time":"2023-10-06T00:00:00Z","timestamp":1696550400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Comput. Surv."],"published-print":{"date-parts":[[2024,3,31]]},"abstract":"<jats:p>Methods to classify objects into two or more classes are at the core of various disciplines. When a set of objects with their true classes is available, a supervised classifier can be trained and employed to decide if, for example, a new patient has cancer or not. The choice of performance measure is critical in deciding which supervised method to use in any particular classification problem. Different measures can lead to very different choices, so the measure should match the objectives. Many performance measures have been developed, and one of them is the F-measure, the harmonic mean of precision and recall. Originally proposed in information retrieval, the F-measure has gained increasing interest in the context of classification. However, the rationale underlying this measure appears weak, and unlike other measures, it does not have a representational meaning. The use of the harmonic mean also has little theoretical justification. The F-measure also stresses one class, which seems inappropriate for general classification problems. We provide a history of the F-measure and its use in computational disciplines, describe its properties, and discuss criticism about the F-Measure. We conclude with alternatives to the F-measure, and recommendations of how to use it effectively.<\/jats:p>","DOI":"10.1145\/3606367","type":"journal-article","created":{"date-parts":[[2023,6,28]],"date-time":"2023-06-28T08:10:44Z","timestamp":1687939844000},"page":"1-24","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":227,"title":["A Review of the F-Measure: Its History, Properties, Criticism, and Alternatives"],"prefix":"10.1145","volume":"56","author":[{"ORCID":"https:\/\/orcid.org\/0000-0003-3435-2015","authenticated-orcid":false,"given":"Peter","family":"Christen","sequence":"first","affiliation":[{"name":"The Australian National University, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-4649-5622","authenticated-orcid":false,"given":"David J.","family":"Hand","sequence":"additional","affiliation":[{"name":"Imperial College London, UK"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-6503-0302","authenticated-orcid":false,"given":"Nishadi","family":"Kirielle","sequence":"additional","affiliation":[{"name":"The Australian National University, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2023,10,6]]},"reference":[{"key":"e_1_3_3_2_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2009.5206596"},{"issue":"3","key":"e_1_3_3_3_2","first-page":"204","article-title":"Evaluation of some coefficients for use in numerical taxonomy of microorganisms","volume":"27","author":"Austin Brian","year":"1977","unstructured":"Brian Austin and Rita R. Colwell. 1977. Evaluation of some coefficients for use in numerical taxonomy of microorganisms. International Journal of Systematic and Evolutionary Microbiology 27, 3 (1977), 204\u2013210.","journal-title":"International Journal of Systematic and Evolutionary Microbiology"},{"key":"e_1_3_3_4_2","doi-asserted-by":"publisher","DOI":"10.5555\/2876198"},{"key":"e_1_3_3_5_2","volume-title":"Theoretical and empirical models","author":"Benton Thomas","year":"2001","unstructured":"Thomas Benton. 2001. Theoretical and empirical models. Ph. D. Dissertation. Department of Mathematics, Imperial College, London."},{"key":"e_1_3_3_6_2","doi-asserted-by":"publisher","DOI":"10.1126\/sciadv.abi8021"},{"key":"e_1_3_3_7_2","first-page":"451","volume-title":"Joint European Conference on Machine Learning and Knowledge Discovery in Databases","author":"Boyd Kendrick","year":"2013","unstructured":"Kendrick Boyd, Kevin H. Eng, and C. David Page. 2013. Area under the precision-recall curve: Point estimates and confidence intervals. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Springer, 451\u2013466."},{"key":"e_1_3_3_8_2","doi-asserted-by":"publisher","DOI":"10.1145\/2907070"},{"key":"e_1_3_3_9_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2011.5995508"},{"key":"e_1_3_3_10_2","volume-title":"Loss Functions for Binary Class Probability Estimation and Classification: Structure and Applications","author":"Buja Andreas","year":"2005","unstructured":"Andreas Buja, Werner Stuetzle, and Yi Shen. 2005. Loss Functions for Binary Class Probability Estimation and Classification: Structure and Applications. Technical Report. The Wharton School, University of Pennsylvania."},{"key":"e_1_3_3_11_2","doi-asserted-by":"publisher","DOI":"10.1613\/jair.953"},{"key":"e_1_3_3_12_2","doi-asserted-by":"publisher","DOI":"10.1186\/s12864-019-6413-7"},{"key":"e_1_3_3_13_2","doi-asserted-by":"publisher","DOI":"10.3115\/1072064.1072067"},{"key":"e_1_3_3_14_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-59706-1"},{"key":"e_1_3_3_15_2","doi-asserted-by":"publisher","DOI":"10.1145\/1143844.1143874"},{"key":"e_1_3_3_16_2","first-page":"1","article-title":"Statistical comparisons of classifiers over multiple data sets","volume":"7","author":"Dem\u0161ar Janez","year":"2006","unstructured":"Janez Dem\u0161ar. 2006. Statistical comparisons of classifiers over multiple data sets. Journal of Machine Learning Research 7, Jan (2006), 1\u201330.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_3_17_2","doi-asserted-by":"publisher","DOI":"10.1016\/0167-8655(95)00113-1"},{"key":"e_1_3_3_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2010.5540041"},{"key":"e_1_3_3_19_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2005.10.010"},{"key":"e_1_3_3_20_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.patrec.2008.08.010"},{"key":"e_1_3_3_21_2","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1983.10478008"},{"key":"e_1_3_3_22_2","volume-title":"Data Mining: Concepts and Techniques (1st ed.)","author":"Han Jiawei","year":"2000","unstructured":"Jiawei Han and Micheline Kamber. 2000. Data Mining: Concepts and Techniques (1st ed.). Morgan Kaufmann, San Francisco."},{"key":"e_1_3_3_23_2","doi-asserted-by":"publisher","DOI":"10.2307\/2983326"},{"key":"e_1_3_3_24_2","volume-title":"Construction and Assessment of Classification Rules","author":"Hand David J.","year":"1997","unstructured":"David J. Hand. 1997. Construction and Assessment of Classification Rules. John Wiley and Sons, Chichester."},{"key":"e_1_3_3_25_2","volume-title":"Measurement Theory and Practice: The World Through Quantification","author":"Hand David J.","year":"2004","unstructured":"David J. Hand. 2004. Measurement Theory and Practice: The World Through Quantification. Edward Arnold, London."},{"key":"e_1_3_3_26_2","doi-asserted-by":"publisher","DOI":"10.1214\/088342306000000060"},{"key":"e_1_3_3_27_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-009-5119-5"},{"key":"e_1_3_3_28_2","doi-asserted-by":"publisher","DOI":"10.1111\/j.1751-5823.2012.00183.x"},{"key":"e_1_3_3_29_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11634-021-00490-3"},{"key":"e_1_3_3_30_2","doi-asserted-by":"publisher","DOI":"10.1007\/s11222-017-9746-6"},{"key":"e_1_3_3_31_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10994-021-05964-1"},{"key":"e_1_3_3_32_2","doi-asserted-by":"publisher","DOI":"10.5555\/500820"},{"key":"e_1_3_3_33_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-0-387-84858-7"},{"key":"e_1_3_3_34_2","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2008.239"},{"key":"e_1_3_3_35_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-74025-2"},{"key":"e_1_3_3_36_2","first-page":"223","article-title":"Nouvelles recherches sur la distribution florale","volume":"44","author":"Jaccard Paul","year":"1908","unstructured":"Paul Jaccard. 1908. Nouvelles recherches sur la distribution florale. Bulletin de la Soci\u00e9t\u00e9 Vaudoise des Sciences Naturelles 44, 163 (1908), 223\u2013270.","journal-title":"Bulletin de la Soci\u00e9t\u00e9 Vaudoise des Sciences Naturelles"},{"key":"e_1_3_3_37_2","doi-asserted-by":"publisher","DOI":"10.4103\/2153-3539.186902"},{"key":"e_1_3_3_38_2","doi-asserted-by":"publisher","DOI":"10.1145\/1102351.1102399"},{"key":"e_1_3_3_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/3470918"},{"key":"e_1_3_3_40_2","doi-asserted-by":"publisher","DOI":"10.1145\/360402.360406"},{"key":"e_1_3_3_41_2","doi-asserted-by":"publisher","DOI":"10.5555\/1594916"},{"key":"e_1_3_3_42_2","doi-asserted-by":"publisher","DOI":"10.1145\/215206.215366"},{"key":"e_1_3_3_43_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-662-44851-9_15"},{"key":"e_1_3_3_44_2","volume-title":"Detection Theory: A User\u2019s Guide (2nd ed.)","author":"Macmillan Neil A.","year":"2005","unstructured":"Neil A. Macmillan and C. Douglas Creelman. 2005. Detection Theory: A User\u2019s Guide (2nd ed.). Lawrence Erlbaum Associates, New York."},{"key":"e_1_3_3_45_2","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511809071"},{"key":"e_1_3_3_46_2","first-page":"1050","volume-title":"International Joint Conference on Artificial Intelligence","author":"McCarthy Joseph F.","year":"1995","unstructured":"Joseph F. McCarthy and Wendy G. Lehnert. 1995. Using decision trees for coreference resolution. In International Joint Conference on Artificial Intelligence. AAAI, 1050\u20131055."},{"key":"e_1_3_3_47_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10618-012-0295-5"},{"key":"e_1_3_3_48_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10699-018-9567-x"},{"key":"e_1_3_3_49_2","doi-asserted-by":"publisher","DOI":"10.1145\/3439726"},{"key":"e_1_3_3_50_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.eswa.2016.03.045"},{"key":"e_1_3_3_51_2","doi-asserted-by":"publisher","DOI":"10.1007\/s13042-011-0012-5"},{"key":"e_1_3_3_52_2","unstructured":"David M. W. Powers. 2019. What the F-measure doesn\u2019t measure: Features flaws fallacies and fixes. https:\/\/arxiv.org\/abs\/1503.06410v2"},{"key":"e_1_3_3_53_2","doi-asserted-by":"crossref","first-page":"315","DOI":"10.1007\/978-3-642-23166-7_12","volume-title":"Data Mining: Foundations and Intelligent Paradigms","author":"Raeder Troy","year":"2012","unstructured":"Troy Raeder, George Forman, and Nitesh V. Chawla. 2012. Learning from imbalanced data: Evaluation matters. In Data Mining: Foundations and Intelligent Paradigms. Springer, Berlin, 315\u2013331."},{"key":"e_1_3_3_54_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2019.00075"},{"key":"e_1_3_3_55_2","volume-title":"Introduction to Modern Information Retrieval","author":"Salton Gerard","year":"1983","unstructured":"Gerard Salton and Michael J. McGill. 1983. Introduction to Modern Information Retrieval. McGraw-Hill, Singapore."},{"key":"e_1_3_3_56_2","article-title":"The truth of the F-measure","author":"Sasaki Yutaka","year":"2007","unstructured":"Yutaka Sasaki. 2007. The truth of the F-measure. University of Manchester, MIB-School of Computer Science.","journal-title":"University of Manchester, MIB-School of Computer Science"},{"key":"e_1_3_3_57_2","volume-title":"Numerical Taxonomy","author":"Sokal Robert R.","year":"1963","unstructured":"Robert R. Sokal and Peter H. A. Sneath. 1963. Numerical Taxonomy. W.H. Freeman and Co., San Francisco."},{"key":"e_1_3_3_58_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.ipm.2009.03.002"},{"key":"e_1_3_3_59_2","volume-title":"A Method of Establishing Groups of Equal Amplitude in Plant Sociology based on Similarity of Species Content and Its Application to Analyses of the Vegetation on Danish Commons","author":"S\u00f8rensen Thorvald Julius","year":"1948","unstructured":"Thorvald Julius S\u00f8rensen. 1948. A Method of Establishing Groups of Equal Amplitude in Plant Sociology based on Similarity of Species Content and Its Application to Analyses of the Vegetation on Danish Commons. I Kommission Hos E. Munksgaard, Copenhagen."},{"key":"e_1_3_3_60_2","doi-asserted-by":"publisher","DOI":"10.5555\/539927"},{"key":"e_1_3_3_61_2","unstructured":"Adam Yedidia. 2016. Against the F-score. Retrieved from https:\/\/adamyedidia.files.wordpress.com\/2014\/11\/f_score.pdf. Blogpost accessed 12 April 2023."}],"container-title":["ACM Computing Surveys"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3606367","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3606367","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T12:47:08Z","timestamp":1750164428000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3606367"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,10,6]]},"references-count":60,"journal-issue":{"issue":"3","published-print":{"date-parts":[[2024,3,31]]}},"alternative-id":["10.1145\/3606367"],"URL":"https:\/\/doi.org\/10.1145\/3606367","relation":{},"ISSN":["0360-0300","1557-7341"],"issn-type":[{"value":"0360-0300","type":"print"},{"value":"1557-7341","type":"electronic"}],"subject":[],"published":{"date-parts":[[2023,10,6]]},"assertion":[{"value":"2022-06-09","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-06-02","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2023-10-06","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}