{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,4,4]],"date-time":"2026-04-04T18:10:16Z","timestamp":1775326216439,"version":"3.50.1"},"reference-count":86,"publisher":"Association for Computing Machinery (ACM)","issue":"8","content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["Proc. VLDB Endow."],"published-print":{"date-parts":[[2024,4]]},"abstract":"<jats:p>Federated Learning (FL) provides a privacy-preserving and decentralized approach to collaborative machine learning for multiple FL clients. The contribution estimation mechanism in FL is extensively studied within the database community, which aims to compute fair and reasonable contribution scores as incentives to motivate FL clients. However, designing such methods involves challenges in three aspects: effectiveness, robustness, and efficiency. Firstly, contribution estimation methods should utilize the data utility information of various client coalitions rather than that of individual clients to ensure effectiveness. Secondly, we should beware of adverse clients who may exploit tactics like data replication or label flipping. Thirdly, estimating contribution in FL can be time-consuming due to enumerating various client coalitions.<\/jats:p><jats:p>Despite numerous proposed methods to address these challenges, each possesses distinct advantages and limitations based on specific settings. However, existing methods have yet to be thoroughly evaluated and compared in the same experimental framework. Therefore, a unified and comprehensive evaluation framework is necessary to compare these methods under the same experimental settings. This paper conducts an extensive survey of contribution estimation methods in FL and introduces a comprehensive framework to evaluate their effectiveness, robustness, and efficiency. Through empirical results, we present extensive observations, valuable discoveries, and an adaptable testing framework that can facilitate future research in designing and evaluating contribution estimation methods in FL.<\/jats:p>","DOI":"10.14778\/3659437.3659459","type":"journal-article","created":{"date-parts":[[2024,5,31]],"date-time":"2024-05-31T16:22:27Z","timestamp":1717172547000},"page":"2077-2090","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":12,"title":["Contributions Estimation in Federated Learning: A Comprehensive Experimental Evaluation"],"prefix":"10.14778","volume":"17","author":[{"given":"Yiwei","family":"Chen","sequence":"first","affiliation":[{"name":"Tsinghua University"}]},{"given":"Kaiyu","family":"Li","sequence":"additional","affiliation":[{"name":"Tsinghua University"}]},{"given":"Guoliang","family":"Li","sequence":"additional","affiliation":[{"name":"Tsinghua University, Zhongguancun Lab"}]},{"given":"Yong","family":"Wang","sequence":"additional","affiliation":[{"name":"Tsinghua University"}]}],"member":"320","published-online":{"date-parts":[[2024,5,31]]},"reference":[{"key":"e_1_2_1_1_1","volume-title":"Felix Xinnan X Yu, Sanjiv Kumar, and Brendan McMahan.","author":"Agarwal Naman","year":"2018","unstructured":"Naman Agarwal, Ananda Theertha Suresh, Felix Xinnan X Yu, Sanjiv Kumar, and Brendan McMahan. 2018. cpSGD: Communication-efficient and differentially-private distributed SGD. Advances in Neural Information Processing Systems 31 (2018)."},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","unstructured":"David Aha. 1991. Tic-Tac-Toe Endgame. UCI Machine Learning Repository. 10.24432\/C5688J","DOI":"10.24432\/C5688J"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.24432\/C5XW20"},{"key":"e_1_2_1_4_1","volume-title":"Pattern recognition and machine learning","author":"Bishop Christopher M","unstructured":"Christopher M Bishop and Nasser M Nasrabadi. 2006. Pattern recognition and machine learning. Vol. 4. Springer."},{"key":"e_1_2_1_5_1","series-title":"SIAM review 60, 2","volume-title":"Optimization methods for large-scale machine learning","author":"Bottou L\u00e9on","year":"2018","unstructured":"L\u00e9on Bottou, Frank E Curtis, and Jorge Nocedal. 2018. Optimization methods for large-scale machine learning. SIAM review 60, 2 (2018), 223--311."},{"key":"e_1_2_1_6_1","volume-title":"Models in cooperative game theory","author":"Branzei Rodica","unstructured":"Rodica Branzei, Dinko Dimitrov, and Stef Tijs. 2008. Models in cooperative game theory. Vol. 556. Springer Science & Business Media."},{"key":"e_1_2_1_7_1","volume-title":"Coefficient of Determination (R 2)","author":"Bucchianico Alessandro Di","year":"2014","unstructured":"Alessandro Di Bucchianico. 2014. Coefficient of Determination (R 2). Wiley StatsRef: Statistics Reference Online (2014)."},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.3390\/electronics8080832"},{"key":"e_1_2_1_9_1","volume-title":"Root mean square error (RMSE) or mean absolute error (MAE)?-Arguments against avoiding RMSE in the literature. Geoscientific model development 7, 3","author":"Chai Tianfeng","year":"2014","unstructured":"Tianfeng Chai and Roland R Draxler. 2014. Root mean square error (RMSE) or mean absolute error (MAE)?-Arguments against avoiding RMSE in the literature. Geoscientific model development 7, 3 (2014), 1247--1250."},{"key":"e_1_2_1_10_1","volume-title":"Dealing with label quality disparity in federated learning","author":"Chen Yiqiang","year":"2020","unstructured":"Yiqiang Chen, Xiaodong Yang, Xin Qin, Han Yu, Piu Chan, and Zhiqi Shen. 2020. Dealing with label quality disparity in federated learning. Federated Learning: Privacy and Incentive (2020), 108--121."},{"key":"e_1_2_1_11_1","volume-title":"Data Pricing in Machine Learning Pipelines. arXiv preprint arXiv:2108.07915","author":"Cong Zicun","year":"2021","unstructured":"Zicun Cong, Xuan Luo, Pei Jian, Feida Zhu, and Yong Zhang. 2021. Data Pricing in Machine Learning Pipelines. arXiv preprint arXiv:2108.07915 (2021)."},{"key":"e_1_2_1_12_1","volume-title":"Kaggle. Retrieved","author":"Elgiriyewithana Nidula","year":"2023","unstructured":"Nidula Elgiriyewithana. 2023. Credit Card Fraud Detection Dataset 2023, Version 1. Kaggle. Retrieved February 11, 2024 from https:\/\/www.kaggle.com\/datasets\/nelgiriyewithana\/credit-card-fraud-detection-dataset-2023\/version\/1"},{"key":"e_1_2_1_13_1","volume-title":"Improving Fairness for Data Valuation in Horizontal Federated Learning. arXiv preprint arXiv:2109.09046","author":"Fan Zhenan","year":"2021","unstructured":"Zhenan Fan, Huang Fang, Zirui Zhou, Jian Pei, Michael P Friedlander, Changxin Liu, and Yong Zhang. 2021. Improving Fairness for Data Valuation in Horizontal Federated Learning. arXiv preprint arXiv:2109.09046 (2021)."},{"key":"e_1_2_1_14_1","volume-title":"Fair and efficient contribution valuation for vertical federated learning. arXiv preprint arXiv:2201.02658","author":"Fan Zhenan","year":"2022","unstructured":"Zhenan Fan, Huang Fang, Zirui Zhou, Jian Pei, Michael P Friedlander, and Yong Zhang. 2022. Fair and efficient contribution valuation for vertical federated learning. arXiv preprint arXiv:2201.02658 (2022)."},{"key":"e_1_2_1_15_1","volume-title":"Dong In Kim, and Ying-Chang Liang","author":"Feng Shaohan","year":"2019","unstructured":"Shaohan Feng, Dusit Niyato, Ping Wang, Dong In Kim, and Ying-Chang Liang. 2019. Joint service pricing and cooperative relay communication for federated learning. In 2019 International Conference on Internet of Things (iThings) and IEEE Green Computing and Communications (GreenCom) and IEEE Cyber, Physical and Social Computing (CPSCom) and IEEE Smart Data (SmartData). IEEE, 815--820."},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.14778\/3407790.3407800"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR52688.2022.00987"},{"key":"e_1_2_1_18_1","volume-title":"International Conference on Machine Learning. PMLR, 3535--3544","author":"Ghorbani Amirata","year":"2020","unstructured":"Amirata Ghorbani, Michael Kim, and James Zou. 2020. A distributional framework for data valuation. In International Conference on Machine Learning. PMLR, 3535--3544."},{"key":"e_1_2_1_19_1","volume-title":"International Conference on Machine Learning. PMLR, 2242--2251","author":"Ghorbani Amirata","year":"2019","unstructured":"Amirata Ghorbani and James Zou. 2019. Data shapley: Equitable valuation of data for machine learning. In International Conference on Machine Learning. PMLR, 2242--2251."},{"key":"e_1_2_1_20_1","first-page":"2379","article-title":"Worker-contributed data utility measurement for visual crowdsensing systems","volume":"16","author":"Guo Bin","year":"2016","unstructured":"Bin Guo, Huihui Chen, Qi Han, Zhiwen Yu, Daqing Zhang, and Yu Wang. 2016. Worker-contributed data utility measurement for visual crowdsensing systems. IEEE Transactions on Mobile Computing 16, 8 (2016), 2379--2391.","journal-title":"IEEE Transactions on Mobile Computing"},{"key":"e_1_2_1_21_1","volume-title":"Data valuation for vertical federated learning: An information-theoretic approach. arXiv preprint arXiv:2112.08364","author":"Han Xiao","year":"2021","unstructured":"Xiao Han, Leye Wang, and Junjie Wu. 2021. Data valuation for vertical federated learning: An information-theoretic approach. arXiv preprint arXiv:2112.08364 (2021)."},{"key":"e_1_2_1_22_1","volume-title":"Promoting Collaborations in Cross-Silo Federated Learning: Challenges and Opportunities","author":"Huang Chao","year":"2023","unstructured":"Chao Huang, Ming Tang, Qian Ma, Jianwei Huang, and Xin Liu. 2023. Promoting Collaborations in Cross-Silo Federated Learning: Challenges and Opportunities. IEEE Communications Magazine (2023)."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/TPS-ISA50397.2020.00014"},{"key":"e_1_2_1_24_1","volume-title":"Facing imbalanced data-recommendations for the use of performance metrics. In 2013 Humaine association conference on affective computing and intelligent interaction","author":"Jeni L\u00e1szl\u00f3 A","unstructured":"L\u00e1szl\u00f3 A Jeni, Jeffrey F Cohn, and Fernando De La Torre. 2013. Facing imbalanced data-recommendations for the use of performance metrics. In 2013 Humaine association conference on affective computing and intelligent interaction. IEEE, 245--251."},{"key":"e_1_2_1_25_1","volume-title":"Nezihe Merve Gurel, Bo Li, Ce Zhang, Costas J Spanos, and Dawn Song.","author":"Jia Ruoxi","year":"2019","unstructured":"Ruoxi Jia, David Dao, Boxin Wang, Frances Ann Hubis, Nezihe Merve Gurel, Bo Li, Ce Zhang, Costas J Spanos, and Dawn Song. 2019. Efficient task-specific data valuation for nearest neighbor algorithms. arXiv preprint arXiv:1908.08619 (2019)."},{"key":"e_1_2_1_26_1","volume-title":"The 22nd International Conference on Artificial Intelligence and Statistics. PMLR, 1167--1176","author":"Jia Ruoxi","year":"2019","unstructured":"Ruoxi Jia, David Dao, Boxin Wang, Frances Ann Hubis, Nick Hynes, Nezihe Merve G\u00fcrel, Bo Li, Ce Zhang, Dawn Song, and Costas J Spanos. 2019. Towards efficient data valuation based on the shapley value. In The 22nd International Conference on Artificial Intelligence and Statistics. PMLR, 1167--1176."},{"key":"e_1_2_1_27_1","volume-title":"Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 8239--8247","author":"Jia Ruoxi","year":"2021","unstructured":"Ruoxi Jia, Fan Wu, Xuehui Sun, Jiacen Xu, David Dao, Bhavya Kailkhura, Ce Zhang, Bo Li, and Dawn Song. 2021. Scalability vs. utility: Do we have to sacrifice one for the other in data importance quantification?. In Proceedings of the IEEE\/CVF Conference on Computer Vision and Pattern Recognition. 8239--8247."},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/3187009.3177733"},{"key":"e_1_2_1_29_1","volume-title":"Retrieved","author":"Kaggle Kaggle","year":"2024","unstructured":"Kaggle [n.d.]. Kaggle: Your Machine Learning and Data Science Community. Retrieved February 14, 2024 from https:\/\/www.kaggle.com"},{"key":"e_1_2_1_30_1","volume-title":"Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al.","author":"Kairouz Peter","year":"2021","unstructured":"Peter Kairouz, H Brendan McMahan, Brendan Avent, Aur\u00e9lien Bellet, Mehdi Bennis, Arjun Nitin Bhagoji, Kallista Bonawitz, Zachary Charles, Graham Cormode, Rachel Cummings, et al. 2021. Advances and open problems in federated learning. Foundations and Trends\u00ae in Machine Learning 14, 1--2 (2021), 1--210."},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/JIOT.2019.2940820"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1162\/089976699300016304"},{"key":"e_1_2_1_33_1","volume-title":"The UCI Machine Learning Repository. Retrieved","author":"Kelly Markelle","year":"2023","unstructured":"Markelle Kelly, Rachel Longjohn, and Kolby Nottingham. [n.d.]. The UCI Machine Learning Repository. Retrieved Oct 1, 2023 from https:\/\/archive.ics.uci.edu"},{"key":"e_1_2_1_34_1","volume-title":"International conference on machine learning. PMLR","author":"Koh Pang Wei","year":"2017","unstructured":"Pang Wei Koh and Percy Liang. 2017. Understanding black-box predictions via influence functions. In International conference on machine learning. PMLR, 1885--1894."},{"key":"e_1_2_1_35_1","first-page":"202","article-title":"Scaling up the accuracy of naive-bayes classifiers: A decision-tree hybrid","volume":"96","author":"Ron Kohavi","year":"1996","unstructured":"Ron Kohavi et al. 1996. Scaling up the accuracy of naive-bayes classifiers: A decision-tree hybrid.. In Kdd, Vol. 96. 202--207.","journal-title":"Kdd"},{"key":"e_1_2_1_36_1","volume-title":"Ananda Theertha Suresh, and Dave Bacon","author":"Kone\u010dn\u1ef3 Jakub","year":"2016","unstructured":"Jakub Kone\u010dn\u1ef3, H Brendan McMahan, Felix X Yu, Peter Richt\u00e1rik, Ananda Theertha Suresh, and Dave Bacon. 2016. Federated learning: Strategies for improving communication efficiency. arXiv preprint arXiv:1610.05492 (2016)."},{"key":"e_1_2_1_37_1","volume-title":"International Conference on Artificial Intelligence and Statistics. PMLR, 793--801","author":"Kwon Yongchan","year":"2021","unstructured":"Yongchan Kwon, Manuel A Rivas, and James Zou. 2021. Efficient computation and analysis of distributional shapley values. In International Conference on Artificial Intelligence and Statistics. PMLR, 793--801."},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1109\/TIFS.2023.3295949"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDE53745.2022.00077"},{"key":"e_1_2_1_40_1","volume-title":"Abnormal Client Behavior Detection in Federated Learning. CoRR abs\/1910.09933","author":"Li Suyi","year":"2019","unstructured":"Suyi Li, Yong Cheng, Yang Liu, Wei Wang, and Tianjian Chen. 2019. Abnormal Client Behavior Detection in Federated Learning. CoRR abs\/1910.09933 (2019). arXiv:1910.09933 http:\/\/arxiv.org\/abs\/1910.09933"},{"key":"e_1_2_1_41_1","volume-title":"Ameet Talwalkar, and Virginia Smith.","author":"Li Tian","year":"2020","unstructured":"Tian Li, Anit Kumar Sahu, Ameet Talwalkar, and Virginia Smith. 2020. Federated learning: Challenges, methods, and future directions. IEEE signal processing magazine 37, 3 (2020), 50--60."},{"key":"e_1_2_1_42_1","volume-title":"GTG-Shapley: Efficient and Accurate Participant Contribution Evaluation in Federated Learning. arXiv preprint arXiv:2109.02053","author":"Liu Zelei","year":"2021","unstructured":"Zelei Liu, Yuanyuan Chen, Han Yu, Yang Liu, and Lizhen Cui. 2021. GTG-Shapley: Efficient and Accurate Participant Contribution Evaluation in Federated Learning. arXiv preprint arXiv:2109.02053 (2021)."},{"key":"e_1_2_1_43_1","volume-title":"On shapley value in data assemblage under independent utility. arXiv preprint arXiv:2208.01163","author":"Luo Xuan","year":"2022","unstructured":"Xuan Luo, Jian Pei, Zicun Cong, and Cheng Xu. 2022. On shapley value in data assemblage under independent utility. arXiv preprint arXiv:2208.01163 (2022)."},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","DOI":"10.23919\/WiOpt52861.2021.9589136"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-030-63076-8_14"},{"key":"e_1_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDEW53142.2021.00023"},{"key":"e_1_2_1_47_1","unstructured":"Brendan McMahan Eider Moore Daniel Ramage Seth Hampson and Blaise Aguera y Arcas. 2017. Communication-efficient learning of deep networks from decentralized data. In Artificial intelligence and statistics. PMLR 1273--1282."},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.dss.2014.03.001"},{"key":"e_1_2_1_49_1","unstructured":"Sergio Moro Raul Laureano and Paulo Cortez. 2011. Using data mining for bank direct marketing: An application of the crisp-dm methodology. (2011)."},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","unstructured":"S. Moro P. Rita and P. Cortez. 2012. Bank Marketing. UCI Machine Learning Repository. 10.24432\/C5K306","DOI":"10.24432\/C5K306"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.orl.2014.11.001"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1109\/TWC.2020.2971981"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2020.3045927"},{"key":"e_1_2_1_54_1","volume-title":"Introduction to the theory of cooperative games","author":"Peleg Bezalel","unstructured":"Bezalel Peleg and Peter Sudh\u00f6lter. 2007. Introduction to the theory of cooperative games. Vol. 34. Springer Science & Business Media."},{"key":"e_1_2_1_55_1","volume-title":"Rewarding high-quality data via influence functions. arXiv preprint arXiv:1908.11598","author":"Richardson Adam","year":"2019","unstructured":"Adam Richardson, Aris Filos-Ratsikas, and Boi Faltings. 2019. Rewarding high-quality data via influence functions. arXiv preprint arXiv:1908.11598 (2019)."},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.1017\/CBO9780511528446"},{"key":"e_1_2_1_57_1","volume-title":"The shapley value in machine learning. arXiv preprint arXiv:2202.05594","author":"Rozemberczki Benedek","year":"2022","unstructured":"Benedek Rozemberczki, Lauren Watson, P\u00e9ter Bayer, Hao-Tsung Yang, Oliv\u00e9r Kiss, Sebastian Nilsson, and Rik Sarkar. 2022. The shapley value in machine learning. arXiv preprint arXiv:2202.05594 (2022)."},{"key":"e_1_2_1_58_1","series-title":"SIAM Journal on applied mathematics 17, 6","volume-title":"The nucleolus of a characteristic function game","author":"Schmeidler David","year":"1969","unstructured":"David Schmeidler. 1969. The nucleolus of a characteristic function game. SIAM Journal on applied mathematics 17, 6 (1969), 1163--1170."},{"key":"e_1_2_1_59_1","doi-asserted-by":"crossref","unstructured":"Lloyd S Shapley et al. 1953. A value for n-person games. (1953).","DOI":"10.1515\/9781400881970-018"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","DOI":"10.1109\/TNNLS.2023.3263594"},{"key":"e_1_2_1_61_1","volume-title":"Fedccea: A practical approach of client contribution evaluation for federated learning. arXiv preprint arXiv:2106.02310","author":"Shyn Sung Kuk","year":"2021","unstructured":"Sung Kuk Shyn, Donghee Kim, and Kwangsu Kim. 2021. Fedccea: A practical approach of client contribution evaluation for federated learning. arXiv preprint arXiv:2106.02310 (2021)."},{"key":"e_1_2_1_62_1","volume-title":"International conference on machine learning. PMLR, 8927--8936","author":"Ling Sim Rachael Hwee","year":"2020","unstructured":"Rachael Hwee Ling Sim, Yehong Zhang, Mun Choon Chan, and Bryan Kian Hsiang Low. 2020. Collaborative machine learning with incentive-aware model rewards. In International conference on machine learning. PMLR, 8927--8936."},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1007\/s001820100072"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.1109\/BigData47090.2019.9006327"},{"key":"e_1_2_1_65_1","volume-title":"Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research 15, 1","author":"Srivastava Nitish","year":"2014","unstructured":"Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. The journal of machine learning research 15, 1 (2014), 1929--1958."},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/3580305.3599500"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","unstructured":"Stephen Tridgell. 2016. Dota2 Games Results. UCI Machine Learning Repository. 10.24432\/C5W593","DOI":"10.24432\/C5W593"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1007\/s13278-017-0480-z"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1109\/BigData47090.2019.9006179"},{"key":"e_1_2_1_70_1","volume-title":"A principled approach to data valuation for federated learning","author":"Wang Tianhao","year":"2020","unstructured":"Tianhao Wang, Johannes Rausch, Ce Zhang, Ruoxi Jia, and Dawn Song. 2020. A principled approach to data valuation for federated learning. Federated Learning: Privacy and Incentive (2020), 153--167."},{"key":"e_1_2_1_71_1","doi-asserted-by":"crossref","unstructured":"Yong Wang Kaiyu Li Guoliang Li Yunyan Guo and Zhuo Wan. 2024. Fast Robust and Interpretable Participant Contribution Estimation for Federated Learning. In ICDE.","DOI":"10.1109\/ICDE60146.2024.00182"},{"key":"e_1_2_1_72_1","doi-asserted-by":"publisher","DOI":"10.1186\/s40537-016-0043-6"},{"key":"e_1_2_1_73_1","volume-title":"Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Climate research 30, 1","author":"Willmott Cort J","year":"2005","unstructured":"Cort J Willmott and Kenji Matsuura. 2005. Advantages of the mean absolute error (MAE) over the root mean square error (RMSE) in assessing average model performance. Climate research 30, 1 (2005), 79--82."},{"key":"e_1_2_1_74_1","volume-title":"The interior-point revolution in optimization: history, recent developments, and lasting consequences. Bulletin of the American mathematical society 42, 1","author":"Wright Margaret","year":"2005","unstructured":"Margaret Wright. 2005. The interior-point revolution in optimization: history, recent developments, and lasting consequences. Bulletin of the American mathematical society 42, 1 (2005), 39--56."},{"key":"e_1_2_1_75_1","doi-asserted-by":"publisher","DOI":"10.14778\/3611479.3611531"},{"key":"e_1_2_1_76_1","first-page":"16104","article-title":"Gradient driven rewards to guarantee fairness in collaborative machine learning","volume":"34","author":"Xu Xinyi","year":"2021","unstructured":"Xinyi Xu, Lingjuan Lyu, Xingjun Ma, Chenglin Miao, Chuan Sheng Foo, and Bryan Kian Hsiang Low. 2021. Gradient driven rewards to guarantee fairness in collaborative machine learning. Advances in Neural Information Processing Systems 34 (2021), 16104--16117.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_77_1","first-page":"10837","article-title":"Validation free and replication robust volume-based data valuation","volume":"34","author":"Xu Xinyi","year":"2021","unstructured":"Xinyi Xu, Zhaoxuan Wu, Chuan Sheng Foo, and Bryan Kian Hsiang Low. 2021. Validation free and replication robust volume-based data valuation. Advances in Neural Information Processing Systems 34 (2021), 10837--10848.","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_2_1_78_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v35i6.16721"},{"key":"e_1_2_1_79_1","doi-asserted-by":"publisher","DOI":"10.1145\/3339474"},{"key":"e_1_2_1_80_1","volume-title":"International Conference on Machine Learning. PMLR, 10842--10851","author":"Yoon Jinsung","year":"2020","unstructured":"Jinsung Yoon, Sercan Arik, and Tomas Pfister. 2020. Data valuation using reinforcement learning. In International Conference on Machine Learning. PMLR, 10842--10851."},{"key":"e_1_2_1_81_1","volume-title":"A comprehensive survey of incentive mechanism for federated learning. arXiv preprint arXiv:2106.15406","author":"Zeng Rongfei","year":"2021","unstructured":"Rongfei Zeng, Chao Zeng, Xingwei Wang, Bo Li, and Xiaowen Chu. 2021. A comprehensive survey of incentive mechanism for federated learning. arXiv preprint arXiv:2106.15406 (2021)."},{"key":"e_1_2_1_82_1","doi-asserted-by":"publisher","DOI":"10.1109\/MNET.001.1900286"},{"key":"e_1_2_1_83_1","volume-title":"A survey of data pricing methods. Available at SSRN 3609120","author":"Zhang Mengxiao","year":"2020","unstructured":"Mengxiao Zhang and Fernando Beltr\u00e1n. 2020. A survey of data pricing methods. Available at SSRN 3609120 (2020)."},{"key":"e_1_2_1_84_1","volume-title":"When crowdsensing meets federated learning: Privacy-preserving mobile crowdsensing system. arXiv preprint arXiv:2102.10109","author":"Zhao Bowen","year":"2021","unstructured":"Bowen Zhao, Ximeng Liu, and Wei-neng Chen. 2021. When crowdsensing meets federated learning: Privacy-preserving mobile crowdsensing system. arXiv preprint arXiv:2102.10109 (2021)."},{"key":"e_1_2_1_85_1","volume-title":"ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 3060--3064","author":"Zhao Jie","year":"2021","unstructured":"Jie Zhao, Xinghua Zhu, Jianzong Wang, and Jing Xiao. 2021. Efficient client contribution evaluation for horizontal federated learning. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 3060--3064."},{"key":"e_1_2_1_86_1","doi-asserted-by":"publisher","DOI":"10.14778\/3587136.3587141"}],"container-title":["Proceedings of the VLDB Endowment"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.14778\/3659437.3659459","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2024,11,20]],"date-time":"2024-11-20T21:35:32Z","timestamp":1732138532000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.14778\/3659437.3659459"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2024,4]]},"references-count":86,"journal-issue":{"issue":"8","published-print":{"date-parts":[[2024,4]]}},"alternative-id":["10.14778\/3659437.3659459"],"URL":"https:\/\/doi.org\/10.14778\/3659437.3659459","relation":{},"ISSN":["2150-8097"],"issn-type":[{"value":"2150-8097","type":"print"}],"subject":[],"published":{"date-parts":[[2024,4]]},"assertion":[{"value":"2024-05-31","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}