{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,9,15]],"date-time":"2026-09-15T04:45:08Z","timestamp":1789447508457,"version":"build-2803163510"},"reference-count":42,"publisher":"Association for Computing Machinery (ACM)","issue":"5","license":[{"start":{"date-parts":[[2026,5,20]],"date-time":"2026-05-20T00:00:00Z","timestamp":1779235200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Knowl. Discov. Data"],"published-print":{"date-parts":[[2026,6,30]]},"abstract":"<jats:p>\n                    Most evaluation metrics for binary classification are derived from the confusion matrix, which is inherently non-differentiable because it relies on discrete predictions. This limits their direct use as loss functions in gradient-based learning, creating a mismatch between training objectives and evaluation criteria. To that end, we offer a general-purpose approach,\n                    <jats:italic toggle=\"yes\">AnyLoss<\/jats:italic>\n                    , that transforms any confusion-matrix-based metric into a differentiable loss function.\n                    <jats:italic toggle=\"yes\">AnyLoss<\/jats:italic>\n                    employs a distinct approximation strategy to estimate a specific, targeted metric score for the prediction model. This is followed by a theoretical and practical analysis of the method, which involves conducting extensive experiments with neural network architectures ranging from simple to advanced across diverse data modalities, including tabular, image, and text. The experimental results demonstrate the generality of our new method, which can target any evaluation metrics derived from a confusion matrix, and highlight that it excels at handling imbalanced datasets.\n                  <\/jats:p>","DOI":"10.1145\/3805705","type":"journal-article","created":{"date-parts":[[2026,3,27]],"date-time":"2026-03-27T14:22:54Z","timestamp":1774621374000},"page":"1-40","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":1,"title":["Generation of Loss Functions from Matrix-Based Binary Classification Metrics"],"prefix":"10.1145","volume":"20","author":[{"ORCID":"https:\/\/orcid.org\/0009-0008-9572-0882","authenticated-orcid":false,"given":"Doheon","family":"Han","sequence":"first","affiliation":[{"name":"Computer Science and Engineering, University of Notre Dame, Notre Dame, Indiana, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-4322-1076","authenticated-orcid":false,"given":"Nuno","family":"Moniz","sequence":"additional","affiliation":[{"name":"University of Notre Dame, Notre Dame, Indiana, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-3932-5956","authenticated-orcid":false,"given":"Nitesh V.","family":"Chawla","sequence":"additional","affiliation":[{"name":"Computer Science and Engineering, University of Notre Dame, Notre Dame, Indiana, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,5,20]]},"reference":[{"key":"e_1_3_1_2_2","doi-asserted-by":"publisher","DOI":"10.1145\/2034691.2034742"},{"key":"e_1_3_1_3_2","unstructured":"The UCI KDD Archive. 1999. Reuters-21578 Text Categorization Collection. Retrieved February 18 2025 from https:\/\/kdd.ics.uci.edu\/databases\/reuters21578\/reuters21578.html"},{"issue":"77","key":"e_1_3_1_4_2","first-page":"1","article-title":"Time for a change: A tutorial for comparing multiple classifiers through Bayesian analysis","volume":"18","author":"Benavoli Alessio","year":"2017","unstructured":"Alessio Benavoli, Giorgio Corani, Janez Dem\u0161ar, and Marco Zaffalon. 2017. Time for a change: A tutorial for comparing multiple classifiers through Bayesian analysis. Journal of Machine Learning Research 18, 77 (2017), 1\u201336.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_1_5_2","unstructured":"Gabriel B\u00e9n\u00e9dict Vincent Koops Daan Odijk and Maarten de Rijke. 2021. SigmoidF1: A smooth F1 score surrogate loss for multilabel classification. arXiv:2108.10566. Retrieved from https:\/\/arxiv.org\/abs\/2108.10566"},{"key":"e_1_3_1_6_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.cmpb.2020.105635"},{"key":"e_1_3_1_7_2","article-title":"Online f-measure optimization","volume":"28","author":"Busa-Fekete R\u00f3bert","year":"2015","unstructured":"R\u00f3bert Busa-Fekete, Bal\u00e1zs Sz\u00f6r\u00e9nyi, Krzysztof Dembczynski, and Eyke H\u00fcllermeier. 2015. Online f-measure optimization. In Proceedings of the Advances in Neural Information Processing Systems, Vol. 28.","journal-title":"Proceedings of the Advances in Neural Information Processing Systems"},{"key":"e_1_3_1_8_2","doi-asserted-by":"publisher","DOI":"10.5194\/gmd-7-1247-2014"},{"key":"e_1_3_1_9_2","doi-asserted-by":"publisher","DOI":"10.5555\/1622407.1622416"},{"key":"e_1_3_1_10_2","doi-asserted-by":"publisher","DOI":"10.1007\/s10618-011-0222-1"},{"key":"e_1_3_1_11_2","first-page":"4171","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: human Language Technologies, Volume 1 (Long and Short Papers)","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: human Language Technologies, Volume 1 (Long and Short Papers), 4171\u20134186."},{"key":"e_1_3_1_12_2","unstructured":"Alexey Dosovitskiy Lucas Beyer Alexander Kolesnikov Dirk Weissenborn Xiaohua Zhai Thomas Unterthiner Mostafa Dehghani Matthias Minderer Georg Heigold Sylvain Gelly et al. 2020. An image is worth 16\u2009\u00d7\u200916 words: Transformers for image recognition at scale. arXiv:2010.11929. Retrieved from https:\/\/arxiv.org\/abs\/2010.11929"},{"issue":"4","key":"e_1_3_1_13_2","first-page":"42","article-title":"An overview of classification algorithms for imbalanced datasets","volume":"2","author":"Ganganwar Vaishali","year":"2012","unstructured":"Vaishali Ganganwar. 2012. An overview of classification algorithms for imbalanced datasets. International Journal of Emerging Technology and Advanced Engineering 2, 4 (2012), 42\u201347.","journal-title":"International Journal of Emerging Technology and Advanced Engineering"},{"key":"e_1_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.1145\/103162.103163"},{"key":"e_1_3_1_15_2","first-page":"992","volume-title":"Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining","author":"Han Doheon","year":"2024","unstructured":"Doheon Han, Nuno Moniz, and Nitesh V. Chawla. 2024. AnyLoss: Transforming classification metrics into loss functions. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 992\u20131003."},{"key":"e_1_3_1_16_2","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_3_1_17_2","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_3_1_18_2","first-page":"174","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR)","author":"Huang Chen","year":"2021","unstructured":"Chen Huang, Shuangfei Zhai, Pengsheng Guo, and Josh Susskind. 2021. MetricOpt: Learning to optimize black-box evaluation metrics. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 174\u2013183."},{"key":"e_1_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1007\/BF02987158"},{"key":"e_1_3_1_20_2","doi-asserted-by":"crossref","unstructured":"Katarzyna Janocha and Wojciech Marian Czarnecki. 2017. On loss functions for deep neural networks in classification. arXiv:1702.05659. Retrieved from https:\/\/arxiv.org\/abs\/1702.05659","DOI":"10.4467\/20838476SI.16.004.6185"},{"key":"e_1_3_1_21_2","unstructured":"Kaggle. 2021. Breast Cancer Dataset. Retrieved October 2 2023 from https:\/\/www.kaggle.com\/datasets\/yasserh\/breast-cancer-dataset"},{"key":"e_1_3_1_22_2","unstructured":"Kaggle. 2022. Titanic Huge Dataset\u20141M Passengers. Retrieved May 21 2024 from https:\/\/www.kaggle.com\/datasets\/marcpaulo\/titanic-huge-dataset-1m-passengers"},{"key":"e_1_3_1_23_2","unstructured":"Kaggle. 2023. Credit Card Fraud Detection Dataset 2023. Retrieved October 2 2023 from https:\/\/www.kaggle.com\/datasets\/nelgiriyewithana\/credit-card-fraud-detection-dataset-2023"},{"key":"e_1_3_1_24_2","unstructured":"Kaggle. 2023. Diabetes Prediction Dataset. Retrieved October 2 2023 from https:\/\/www.kaggle.com\/datasets\/iammustafatz\/diabetes-prediction-dataset"},{"key":"e_1_3_1_25_2","unstructured":"Alex Krizhevsky and Geoffrey Hinton. 2009. Learning Multiple Layers of Features from Tiny Images. Master\u2019s thesis University of Tront."},{"key":"e_1_3_1_26_2","first-page":"26","article-title":"The Bayesian new statistics: Two historical trends converge","author":"Kruschke John K.","year":"2015","unstructured":"John K. Kruschke and Torrin M. Liddell. 2015. The Bayesian new statistics: Two historical trends converge. SSRN Electronic Journal 2606016 (2015), 26.","journal-title":"SSRN Electronic Journal"},{"key":"e_1_3_1_27_2","doi-asserted-by":"publisher","DOI":"10.1109\/5.726791"},{"key":"e_1_3_1_28_2","unstructured":"Namgil Lee Heejung Yang and Hojin Yoo. 2021. A surrogate loss function for optimization of F \\({}_{\\beta}\\) score in binary classification with imbalanced data. arXiv:2104.01459. Retrieved from https:\/\/arxiv.org\/abs\/2104.01459"},{"issue":"8","key":"e_1_3_1_29_2","doi-asserted-by":"crossref","first-page":"603","DOI":"10.1038\/nmeth.3945","article-title":"Classification evaluation: It is important to understand both what a classification metric expresses and what it hides","volume":"13","author":"Lever Jake","year":"2016","unstructured":"Jake Lever, Martin Krzywinski, and Naomi Altman. 2016. Classification evaluation: It is important to understand both what a classification metric expresses and what it hides. Nature Methods 13, 8 (2016), 603\u2013605.","journal-title":"Nature Methods"},{"key":"e_1_3_1_30_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICCV.2017.324"},{"key":"e_1_3_1_31_2","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-662-44851-9_15"},{"key":"e_1_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.5555\/2002472.2002491"},{"key":"e_1_3_1_33_2","doi-asserted-by":"crossref","first-page":"108913","DOI":"10.1016\/j.patcog.2022.108913","article-title":"Score-oriented loss (SOL) functions","volume":"132","author":"Marchetti Francesco","year":"2022","unstructured":"Francesco Marchetti, Sabrina Guastavino, Michele Piana, and Cristina Campi. 2022. Score-oriented loss (SOL) functions. Pattern Recognition 132 (2022), 108913.","journal-title":"Pattern Recognition"},{"key":"e_1_3_1_34_2","doi-asserted-by":"publisher","DOI":"10.5555\/541177"},{"key":"e_1_3_1_35_2","doi-asserted-by":"crossref","first-page":"115011","DOI":"10.1016\/j.eswa.2021.115011","article-title":"Automated imbalanced classification via meta-learning","volume":"178","author":"Moniz Nuno","year":"2021","unstructured":"Nuno Moniz and Vitor Cerqueira. 2021. Automated imbalanced classification via meta-learning. Expert Systems with Applications 178 (2021), 115011.","journal-title":"Expert Systems with Applications"},{"key":"e_1_3_1_36_2","unstructured":"Sebastian Raschka. 2018. Model evaluation model selection and algorithm selection in machine learning. arXiv:1811.12808. Retrieved from https:\/\/arxiv.org\/abs\/1811.12808"},{"key":"e_1_3_1_37_2","doi-asserted-by":"publisher","DOI":"10.5555\/823454.823889"},{"key":"e_1_3_1_38_2","unstructured":"Han Xiao Kashif Rasul and Roland Vollgraf. 2017. Fashion-MNIST: A novel image dataset for benchmarking machine learning algorithms. arXiv:1708.07747. Retrieved from https:\/\/arxiv.org\/abs\/1708.07747"},{"key":"e_1_3_1_39_2","doi-asserted-by":"publisher","DOI":"10.1145\/383952.383975"},{"key":"e_1_3_1_40_2","doi-asserted-by":"crossref","first-page":"106911","DOI":"10.1016\/j.engappai.2023.106911","article-title":"Review of resampling techniques for the treatment of imbalanced industrial data classification in equipment condition monitoring","volume":"126","author":"Yuan Yage","year":"2023","unstructured":"Yage Yuan, Jianan Wei, Haisong Huang, Weidong Jiao, Jiaxin Wang, and Hualin Chen. 2023. Review of resampling techniques for the treatment of imbalanced industrial data classification in equipment condition monitoring. Engineering Applications of Artificial Intelligence 126 (2023), 106911.","journal-title":"Engineering Applications of Artificial Intelligence"},{"key":"e_1_3_1_41_2","first-page":"253","volume-title":"Proceedings of the 8th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)","author":"Zhang Ye","year":"2017","unstructured":"Ye Zhang and Byron Wallace. 2017. A sensitivity analysis of (and practitioners\u2019 guide to) convolutional neural networks for sentence classification. In Proceedings of the 8th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). Greg Kondrak and Taro Watanabe (Eds.), Asian Federation of Natural Language Processing, Taipei, 253\u2013263. Retrieved from https:\/\/aclanthology.org\/I17-1026\/"},{"key":"e_1_3_1_42_2","doi-asserted-by":"crossref","first-page":"3447","DOI":"10.1609\/aaai.v35i4.16458","article-title":"Bag of tricks for long-tailed visual recognition with deep convolutional neural networks","volume":"35","author":"Zhang Yongshun","year":"2021","unstructured":"Yongshun Zhang, Xiu-Shen Wei, Boyan Zhou, and Jianxin Wu. 2021. Bag of tricks for long-tailed visual recognition with deep convolutional neural networks. Proceedings of the AAAI Conference on Artificial Intelligence 35 (2021), 3447\u20133455.","journal-title":"Proceedings of the AAAI Conference on Artificial Intelligence"},{"key":"e_1_3_1_43_2","doi-asserted-by":"publisher","DOI":"10.1016\/j.bdr.2015.12.001"}],"container-title":["ACM Transactions on Knowledge Discovery from Data"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3805705","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,5,20]],"date-time":"2026-05-20T13:30:17Z","timestamp":1779283817000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3805705"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,5,20]]},"references-count":42,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2026,6,30]]}},"alternative-id":["10.1145\/3805705"],"URL":"https:\/\/doi.org\/10.1145\/3805705","relation":{},"ISSN":["1556-4681","1556-472X"],"issn-type":[{"value":"1556-4681","type":"print"},{"value":"1556-472X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,5,20]]},"assertion":[{"value":"2025-08-13","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-03-20","order":2,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2026-05-20","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}