{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,13]],"date-time":"2026-07-13T19:54:55Z","timestamp":1783972495850,"version":"3.55.0"},"publisher-location":"New York, NY, USA","reference-count":33,"publisher":"ACM","license":[{"start":{"date-parts":[[2020,4,20]],"date-time":"2020-04-20T00:00:00Z","timestamp":1587340800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2020,4,20]]},"DOI":"10.1145\/3366423.3380154","type":"proceedings-article","created":{"date-parts":[[2020,5,4]],"date-time":"2020-05-04T08:11:44Z","timestamp":1588579904000},"page":"729-739","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":22,"title":["Field-aware Calibration: A Simple and Empirically Strong Method for Reliable Probabilistic Predictions"],"prefix":"10.1145","author":[{"given":"Feiyang","family":"Pan","sequence":"first","affiliation":[{"name":"Institute of Computing Technology Chinese Academy of Sciences"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiang","family":"Ao","sequence":"additional","affiliation":[{"name":"Institute of Computing Technology Chinese Academy of Sciences"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Pingzhong","family":"Tang","sequence":"additional","affiliation":[{"name":"Tsinghua University"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Min","family":"Lu","sequence":"additional","affiliation":[{"name":"Tencent"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Dapeng","family":"Liu","sequence":"additional","affiliation":[{"name":"Tencent"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Lei","family":"Xiao","sequence":"additional","affiliation":[{"name":"Tencent"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Qing","family":"He","sequence":"additional","affiliation":[{"name":"Institute of Computing Technology CAS"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2020,4,20]]},"reference":[{"key":"e_1_3_2_1_2_1","doi-asserted-by":"crossref","unstructured":"Antonio Bella C\u00e8sar Ferri Jos\u00e9 Hern\u00e1ndez-Orallo and Mar\u00eda\u00a0Jos\u00e9 Ram\u00edrez-Quintana. 2010. Calibration of machine learning models. In Handbook of Research on Machine Learning Applications and Trends: Algorithms Methods and Techniques. IGI Global 128\u2013146.  Antonio Bella C\u00e8sar Ferri Jos\u00e9 Hern\u00e1ndez-Orallo and Mar\u00eda\u00a0Jos\u00e9 Ram\u00edrez-Quintana. 2010. Calibration of machine learning models. In Handbook of Research on Machine Learning Applications and Trends: Algorithms Methods and Techniques. IGI Global 128\u2013146.","DOI":"10.4018\/978-1-60566-766-9.ch006"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3269206.3269260"},{"key":"e_1_3_2_1_4_1","unstructured":"Yuri Burda Harri Edwards Deepak Pathak Amos Storkey Trevor Darrell and Alexei\u00a0A Efros. 2018. Large-scale study of curiosity-driven learning. arXiv preprint arXiv:1808.04355(2018).  Yuri Burda Harri Edwards Deepak Pathak Amos Storkey Trevor Darrell and Alexei\u00a0A Efros. 2018. Large-scale study of curiosity-driven learning. arXiv preprint arXiv:1808.04355(2018)."},{"key":"e_1_3_2_1_5_1","volume-title":"Assessing calibration of prognostic risk scores. Statistical methods in medical research 25, 4","author":"Crowson S","year":"2016","unstructured":"Cynthia\u00a0 S Crowson , Elizabeth\u00a0 J Atkinson , and Terry\u00a0 M Therneau . 2016. Assessing calibration of prognostic risk scores. Statistical methods in medical research 25, 4 ( 2016 ), 1692\u20131706. Cynthia\u00a0S Crowson, Elizabeth\u00a0J Atkinson, and Terry\u00a0M Therneau. 2016. Assessing calibration of prognostic risk scores. Statistical methods in medical research 25, 4 (2016), 1692\u20131706."},{"key":"e_1_3_2_1_6_1","first-page":"1","article-title":"The comparison and evaluation of forecasters","volume":"32","author":"DeGroot H","year":"1983","unstructured":"Morris\u00a0 H DeGroot and Stephen\u00a0 E Fienberg . 1983 . The comparison and evaluation of forecasters . Journal of the Royal Statistical Society: Series D (The Statistician) 32 , 1 - 2 (1983), 12\u201322. Morris\u00a0H DeGroot and Stephen\u00a0E Fienberg. 1983. The comparison and evaluation of forecasters. Journal of the Royal Statistical Society: Series D (The Statistician) 32, 1-2(1983), 12\u201322.","journal-title":"Journal of the Royal Statistical Society: Series D (The Statistician)"},{"key":"e_1_3_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.5555\/3305381.3305504"},{"key":"e_1_3_2_1_8_1","unstructured":"Yonatan Geifman Guy Uziel and Ran El-Yaniv. 2018. Bias-Reduced Uncertainty Estimation for Deep Neural Classifiers. arXiv preprint arXiv:1805.08206(2018).  Yonatan Geifman Guy Uziel and Ran El-Yaniv. 2018. Bias-Reduced Uncertainty Estimation for Deep Neural Classifiers. arXiv preprint arXiv:1805.08206(2018)."},{"key":"e_1_3_2_1_9_1","volume-title":"Web-scale bayesian click-through rate prediction for sponsored search advertising in microsoft\u2019s bing search engine","author":"Graepel Thore","unstructured":"Thore Graepel , Joaquin\u00a0Quinonero Candela , Thomas Borchert , and Ralf Herbrich . 2010. Web-scale bayesian click-through rate prediction for sponsored search advertising in microsoft\u2019s bing search engine . Omnipress . Thore Graepel, Joaquin\u00a0Quinonero Candela, Thomas Borchert, and Ralf Herbrich. 2010. Web-scale bayesian click-through rate prediction for sponsored search advertising in microsoft\u2019s bing search engine. Omnipress."},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.5555\/3305381.3305518"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1145\/2648584.2648589"},{"key":"e_1_3_2_1_12_1","volume-title":"Proceedings of the 35th International Conference on Machine Learning(Proceedings of Machine Learning Research), Jennifer Dy and Andreas Krause (Eds.), Vol.\u00a080","author":"Hebert-Johnson Ursula","year":"2018","unstructured":"Ursula Hebert-Johnson , Michael Kim , Omer Reingold , and Guy Rothblum . 2018 . Multicalibration: Calibration for the (Computationally-Identifiable) Masses . In Proceedings of the 35th International Conference on Machine Learning(Proceedings of Machine Learning Research), Jennifer Dy and Andreas Krause (Eds.), Vol.\u00a080 . PMLR, Stockholmsm\u00e4ssan, Stockholm Sweden , 1939\u20131948. Ursula Hebert-Johnson, Michael Kim, Omer Reingold, and Guy Rothblum. 2018. Multicalibration: Calibration for the (Computationally-Identifiable) Masses. In Proceedings of the 35th International Conference on Machine Learning(Proceedings of Machine Learning Research), Jennifer Dy and Andreas Krause (Eds.), Vol.\u00a080. PMLR, Stockholmsm\u00e4ssan, Stockholm Sweden, 1939\u20131948."},{"key":"e_1_3_2_1_13_1","unstructured":"Dan Hendrycks and Kevin Gimpel. 2016. A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136(2016).  Dan Hendrycks and Kevin Gimpel. 2016. A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136(2016)."},{"key":"e_1_3_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICDMW.2011.83"},{"key":"e_1_3_2_1_15_1","volume-title":"Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980(2014).","author":"Kingma P","year":"2014","unstructured":"Diederik\u00a0 P Kingma and Jimmy Ba . 2014 . Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980(2014). Diederik\u00a0P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980(2014)."},{"key":"e_1_3_2_1_16_1","unstructured":"Jon Kleinberg Sendhil Mullainathan and Manish Raghavan. 2016. Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807(2016).  Jon Kleinberg Sendhil Mullainathan and Manish Raghavan. 2016. Inherent trade-offs in the fair determination of risk scores. arXiv preprint arXiv:1609.05807(2016)."},{"key":"e_1_3_2_1_17_1","volume-title":"Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence. 73\u201377","author":"Korb B","year":"1999","unstructured":"Kevin\u00a0 B Korb . 1999 . Calibration and the evaluation of predictive learners . In Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence. 73\u201377 . Kevin\u00a0B Korb. 1999. Calibration and the evaluation of predictive learners. In Proceedings of the Sixteenth International Joint Conference on Artificial Intelligence. 73\u201377."},{"key":"e_1_3_2_1_18_1","volume-title":"Conference on Learning Theory. 1165\u20131189","author":"Kot\u0142owski Wojciech","year":"2016","unstructured":"Wojciech Kot\u0142owski , Wouter\u00a0 M Koolen , and Alan Malek . 2016 . Online isotonic regression . In Conference on Learning Theory. 1165\u20131189 . Wojciech Kot\u0142owski, Wouter\u00a0M Koolen, and Alan Malek. 2016. Online isotonic regression. In Conference on Learning Theory. 1165\u20131189."},{"key":"e_1_3_2_1_19_1","unstructured":"Balaji Lakshminarayanan Alexander Pritzel and Charles Blundell. 2017. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems. 6402\u20136413.  Balaji Lakshminarayanan Alexander Pritzel and Charles Blundell. 2017. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems. 6402\u20136413."},{"key":"e_1_3_2_1_20_1","unstructured":"Kimin Lee Honglak Lee Kibok Lee and Jinwoo Shin. 2017. Training confidence-calibrated classifiers for detecting out-of-distribution samples. arXiv preprint arXiv:1711.09325(2017).  Kimin Lee Honglak Lee Kibok Lee and Jinwoo Shin. 2017. Training confidence-calibrated classifiers for detecting out-of-distribution samples. arXiv preprint arXiv:1711.09325(2017)."},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/2487575.2488200"},{"key":"e_1_3_2_1_22_1","volume-title":"Conference on Fairness, Accountability and Transparency. 107\u2013118","author":"Menon Aditya\u00a0Krishna","year":"2018","unstructured":"Aditya\u00a0Krishna Menon and Robert\u00a0 C Williamson . 2018 . The cost of fairness in binary classification . In Conference on Fairness, Accountability and Transparency. 107\u2013118 . Aditya\u00a0Krishna Menon and Robert\u00a0C Williamson. 2018. The cost of fairness in binary classification. In Conference on Fairness, Accountability and Transparency. 107\u2013118."},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.5555\/2888116.2888120"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.0006-341X.2004.00184.x"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/1102351.1102430"},{"key":"e_1_3_2_1_26_1","unstructured":"Yaniv Ovadia Emily Fertig Jie Ren Zachary Nado D Sculley Sebastian Nowozin Joshua\u00a0V Dillon Balaji Lakshminarayanan and Jasper Snoek. 2019. Can You Trust Your Model\u2019s Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift. arXiv preprint arXiv:1906.02530(2019).  Yaniv Ovadia Emily Fertig Jie Ren Zachary Nado D Sculley Sebastian Nowozin Joshua\u00a0V Dillon Balaji Lakshminarayanan and Jasper Snoek. 2019. Can You Trust Your Model\u2019s Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift. arXiv preprint arXiv:1906.02530(2019)."},{"key":"e_1_3_2_1_27_1","volume-title":"Policy Gradients for Contextual Recommendations. In The World Wide Web Conference","author":"Pan Feiyang","year":"2019","unstructured":"Feiyang Pan , Qingpeng Cai , Pingzhong Tang , Fuzhen Zhuang , and Qing He . 2019 . Policy Gradients for Contextual Recommendations. In The World Wide Web Conference ( San Francisco, CA, USA) (WWW \u201919). Association for Computing Machinery, New York, NY, USA, 1421\u20131431. https:\/\/doi.org\/10.1145\/3308558.3313616 10.1145\/3308558.3313616 Feiyang Pan, Qingpeng Cai, Pingzhong Tang, Fuzhen Zhuang, and Qing He. 2019. Policy Gradients for Contextual Recommendations. In The World Wide Web Conference (San Francisco, CA, USA) (WWW \u201919). Association for Computing Machinery, New York, NY, USA, 1421\u20131431. https:\/\/doi.org\/10.1145\/3308558.3313616"},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v33i01.33014675"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3331184.3331268"},{"key":"e_1_3_2_1_30_1","volume-title":"Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers 10, 3","author":"John Platt","year":"1999","unstructured":"John Platt 1999. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers 10, 3 ( 1999 ), 61\u201374. John Platt 1999. Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. Advances in large margin classifiers 10, 3 (1999), 61\u201374."},{"key":"e_1_3_2_1_31_1","unstructured":"Geoff Pleiss Manish Raghavan Felix Wu Jon Kleinberg and Kilian\u00a0Q Weinberger. 2017. On fairness and calibration. In Advances in Neural Information Processing Systems. 5680\u20135689.  Geoff Pleiss Manish Raghavan Felix Wu Jon Kleinberg and Kilian\u00a0Q Weinberger. 2017. On fairness and calibration. In Advances in Neural Information Processing Systems. 5680\u20135689."},{"key":"e_1_3_2_1_32_1","unstructured":"Carlos Riquelme George Tucker and Jasper Snoek. 2018. Deep bayesian bandits showdown: An empirical comparison of bayesian deep networks for thompson sampling. arXiv preprint arXiv:1802.09127(2018).  Carlos Riquelme George Tucker and Jasper Snoek. 2018. Deep bayesian bandits showdown: An empirical comparison of bayesian deep networks for thompson sampling. arXiv preprint arXiv:1802.09127(2018)."},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/775047.775151"},{"key":"e_1_3_2_1_34_1","unstructured":"Muhammad\u00a0Bilal Zafar Isabel Valera Manuel\u00a0Gomez Rodriguez and Krishna\u00a0P Gummadi. 2015. Fairness constraints: Mechanisms for fair classification. arXiv preprint arXiv:1507.05259(2015).  Muhammad\u00a0Bilal Zafar Isabel Valera Manuel\u00a0Gomez Rodriguez and Krishna\u00a0P Gummadi. 2015. Fairness constraints: Mechanisms for fair classification. arXiv preprint arXiv:1507.05259(2015)."}],"event":{"name":"WWW '20: The Web Conference 2020","location":"Taipei Taiwan","acronym":"WWW '20","sponsor":["SIGWEB ACM Special Interest Group on Hypertext, Hypermedia, and Web"]},"container-title":["Proceedings of The Web Conference 2020"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3366423.3380154","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3366423.3380154","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T22:33:00Z","timestamp":1750199580000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3366423.3380154"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,4,20]]},"references-count":33,"alternative-id":["10.1145\/3366423.3380154","10.1145\/3366423"],"URL":"https:\/\/doi.org\/10.1145\/3366423.3380154","relation":{},"subject":[],"published":{"date-parts":[[2020,4,20]]},"assertion":[{"value":"2020-04-20","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}