{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,27]],"date-time":"2026-02-27T06:22:07Z","timestamp":1772173327172,"version":"3.50.1"},"reference-count":45,"publisher":"Association for Computing Machinery (ACM)","issue":"7","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2024,3]]},"abstract":"<jats:p>The Shapley value provides a unique approach to equitably gauge each player's contribution within a coalition and has extensive applications with various utility functions. In data valuation for machine learning, particularly for classification tasks, using classification accuracy as the utility function has become a de facto standard. However, accuracy can be an imprecise metric, potentially missing finer details crucial for valuation. In this paper, we propose the probability-based Shapley (P-Shapley) value, which leverages predicted probabilities to heighten utility differentiation. Several convex calibration functions are further incorporated for probability calibration. We prove that the P-Shapley value outperforms Shapley values based on accuracy or other coarse metrics in approximation stability and the discrimination of marginal utility change can be further improved by convex calibration functions. Extensive experiments on four real-world datasets demonstrate the effectiveness of our approaches.<\/jats:p>","DOI":"10.14778\/3654621.3654638","type":"journal-article","created":{"date-parts":[[2024,5,30]],"date-time":"2024-05-30T22:21:08Z","timestamp":1717107668000},"page":"1737-1750","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":9,"title":["P-Shapley: Shapley Values on Probabilistic Classifiers"],"prefix":"10.14778","volume":"17","author":[{"given":"Haocheng","family":"Xia","sequence":"first","affiliation":[{"name":"Zhejiang University"}]},{"given":"Xiang","family":"Li","sequence":"additional","affiliation":[{"name":"Zhejiang University"}]},{"given":"Junyuan","family":"Pang","sequence":"additional","affiliation":[{"name":"Zhejiang University"}]},{"given":"Jinfei","family":"Liu","sequence":"additional","affiliation":[{"name":"Zhejiang University, ZJU-Hangzhou Global Scientific and Technological Innovation Center"}]},{"given":"Kui","family":"Ren","sequence":"additional","affiliation":[{"name":"Zhejiang University"}]},{"given":"Li","family":"Xiong","sequence":"additional","affiliation":[{"name":"Emory University"}]}],"member":"320","published-online":{"date-parts":[[2024,5,30]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2207.05811"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3583780.3614864"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3299869.3300078"},{"key":"e_1_2_1_4_1","volume-title":"FedFair: Training Fair Models In Cross-Silo Federated Learning. CoRR abs\/2109.05662","author":"Chu Lingyang","year":"2021","unstructured":"Lingyang Chu, Lanjun Wang, Yanjie Dong, Jian Pei, Zirui Zhou, and Yong Zhang. 2021. FedFair: Training Fair Models In Cross-Silo Federated Learning. CoRR abs\/2109.05662 (2021). arXiv:2109.05662 https:\/\/arxiv.org\/abs\/2109.05662"},{"key":"e_1_2_1_5_1","volume-title":"Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence","author":"Cohen Shay B.","year":"2005","unstructured":"Shay B. Cohen, Eytan Ruppin, and Gideon Dror. 2005. Feature Selection Based on the Shapley Value. In IJCAI-05, Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence, Edinburgh, Scotland, UK, July 30 - August 5, 2005, Leslie Pack Kaelbling and Alessandro Saffiotti (Eds.). Professional Book Center, 665--670. http:\/\/ijcai.org\/Proceedings\/05\/Papers\/0763.pdf"},{"key":"e_1_2_1_6_1","doi-asserted-by":"crossref","first-page":"15","DOI":"10.1080\/00401706.1977.10489493","article-title":"Detection of Influential Observation in Linear Regression","volume":"19","author":"Cook R. Dennis","year":"1977","unstructured":"R. Dennis Cook. 1977. Detection of Influential Observation in Linear Regression. Technometrics 19, 1 (1977), 15--18. http:\/\/www.jstor.org\/stable\/1268249","journal-title":"Technometrics"},{"key":"e_1_2_1_7_1","volume-title":"Elements of information theory","author":"Cover Thomas M","unstructured":"Thomas M Cover. 1999. Elements of information theory. John Wiley & Sons."},{"key":"e_1_2_1_8_1","volume-title":"Tests of calibration and goodness-of-fit in the survival setting. Statistics in medicine 34, 10","author":"Demler Olga V","year":"2015","unstructured":"Olga V Demler, Nina P Paynter, and Nancy R Cook. 2015. Tests of calibration and goodness-of-fit in the survival setting. Statistics in medicine 34, 10 (2015), 1659--1680."},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1287\/MOOR.19.2.257"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.1145\/3514221.3517912"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.14778\/3415478.3415508"},{"key":"e_1_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1016\/J.NEUCOM.2022.06.111"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPS-ISA56441.2022.00017"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1145\/3514221.3517855"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/3307339.3342159"},{"key":"e_1_2_1_16_1","volume-title":"Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13--18","volume":"119","author":"Ghorbani Amirata","year":"2020","unstructured":"Amirata Ghorbani, Michael P. Kim, and James Zou. 2020. A Distributional Framework For Data Valuation. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13--18 July 2020, Virtual Event (Proceedings of Machine Learning Research), Vol. 119. PMLR, 3535--3544. http:\/\/proceedings.mlr.press\/v119\/ghorbani20a.html"},{"key":"e_1_2_1_17_1","volume-title":"Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9--15","volume":"97","author":"Ghorbani Amirata","year":"2019","unstructured":"Amirata Ghorbani and James Y. Zou. 2019. Data Shapley: Equitable Valuation of Data for Machine Learning. In Proceedings of the 36th International Conference on Machine Learning, ICML 2019, 9--15 June 2019, Long Beach, California, USA (Proceedings of Machine Learning Research), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.), Vol. 97. PMLR, 2242--2251. http:\/\/proceedings.mlr.press\/v97\/ghorbani19c.html"},{"key":"e_1_2_1_18_1","unstructured":"Jiawei Han Jian Pei and Hanghang Tong. 2022. Data mining: concepts and techniques. Morgan kaufmann."},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.14778\/3342263.3342637"},{"key":"e_1_2_1_21_1","volume-title":"International Conference on Artificial Intelligence and Statistics, AISTATS 2022","volume":"151","author":"Kwon Yongchan","year":"2022","unstructured":"Yongchan Kwon and James Zou. 2022. Beta Shapley: a Unified and Noise-reduced Data Valuation Framework for Machine Learning. In International Conference on Artificial Intelligence and Statistics, AISTATS 2022, 28--30 March 2022, Virtual Event (Proceedings of Machine Learning Research), Gustau Camps-Valls, Francisco J. R. Ruiz, and Isabel Valera (Eds.), Vol. 151. PMLR, 8780--8802. https:\/\/proceedings.mlr.press\/v151\/kwon22a.html"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/SP46215.2023.10179401"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.14778\/3467861.3467872"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.14778\/3583140.3583146"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.14778\/3476311.3476335"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.14778\/3447689.3447700"},{"key":"e_1_2_1_27_1","volume-title":"Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017","author":"Scott","year":"2017","unstructured":"Scott M. Lundberg and Su-In Lee. 2017. A Unified Approach to Interpreting Model Predictions. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4--9, 2017, Long Beach, CA, USA, Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (Eds.). 4765--4774. https:\/\/proceedings.neurips.cc\/paper\/2017\/hash\/8a20a8621978632d76c43dfd28b67767-Abstract.html"},{"key":"e_1_2_1_28_1","volume-title":"Mish: A Self Regularized Non-Monotonic Neural Activation Function. CoRR abs\/1908.08681","author":"Misra Diganta","year":"2019","unstructured":"Diganta Misra. 2019. Mish: A Self Regularized Non-Monotonic Neural Activation Function. CoRR abs\/1908.08681 (2019). arXiv:1908.08681 http:\/\/arxiv.org\/abs\/1908.08681"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2020.3045927"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.14778\/3611540.3611573"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2303.16963"},{"key":"e_1_2_1_32_1","volume-title":"Le","author":"Ramachandran Prajit","year":"2018","unstructured":"Prajit Ramachandran, Barret Zoph, and Quoc V. Le. 2018. Searching for Activation Functions. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Workshop Track Proceedings. OpenReview.net. https:\/\/openreview.net\/forum?id=Hkuq2EkPf"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3375395.3387664"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.1145\/3555041.3589682"},{"key":"e_1_2_1_35_1","unstructured":"Stephanie Schoch Haifeng Xu and Yangfeng Ji. 2022. CS-Shapley: Class-wise Shapley Values for Data Valuation in Classification. In NeurIPS. http:\/\/papers.nips.cc\/paper_files\/paper\/2022\/hash\/df334022279996b07e0870a629c18857-Abstract-Conference.html"},{"key":"e_1_2_1_36_1","volume-title":"A value for n-person games. Contributions to the Theory of Games 2, 28","author":"Shapley Lloyd S","year":"1953","unstructured":"Lloyd S Shapley. 1953. A value for n-person games. Contributions to the Theory of Games 2, 28 (1953), 307--317."},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/BIGDATA47090.2019.9006327"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.14778\/2168651.2168657"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1145\/2641190.2641198"},{"key":"e_1_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.48550\/ARXIV.2306.07171"},{"key":"e_1_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.14778\/3611479.3611531"},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.14778\/3407790.3407859"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3588728"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.1109\/MIS.2020.3010335"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447548.3470814"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3654621.3654638","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,20]],"date-time":"2024-11-20T19:57:38Z","timestamp":1732132658000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3654621.3654638"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,3]]},"references-count":45,"journal-issue":{"issue":"7","published-print":{"date-parts":[[2024,3]]}},"alternative-id":["10.14778\/3654621.3654638"],"URL":"https:\/\/doi.org\/10.14778\/3654621.3654638","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2024,3]]},"assertion":[{"value":"2024-05-30","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}