{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,12,30]],"date-time":"2025-12-30T17:25:26Z","timestamp":1767115526773,"version":"3.41.0"},"publisher-location":"New York, NY, USA","reference-count":32,"publisher":"ACM","license":[{"start":{"date-parts":[[2023,6,12]],"date-time":"2023-06-12T00:00:00Z","timestamp":1686528000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"NSF (National Science Foundation)","doi-asserted-by":"publisher","award":["FAI-2147212"],"award-info":[{"award-number":["FAI-2147212"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2023,6,12]]},"DOI":"10.1145\/3593013.3593980","type":"proceedings-article","created":{"date-parts":[[2023,6,12]],"date-time":"2023-06-12T14:40:46Z","timestamp":1686580846000},"page":"101-110","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":7,"title":["Reconciling Individual Probability Forecasts\u2731"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0586-0515","authenticated-orcid":false,"given":"Aaron","family":"Roth","sequence":"first","affiliation":[{"name":"University of Pennsylvania, USA"}]},{"ORCID":"https:\/\/orcid.org\/0009-0001-7598-2905","authenticated-orcid":false,"given":"Alexander","family":"Tolbert","sequence":"additional","affiliation":[{"name":"University of Pennsylvania, USA"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1002-2521","authenticated-orcid":false,"given":"Scott","family":"Weinstein","sequence":"additional","affiliation":[{"name":"University of Pennsylvania, USA"}]}],"member":"320","published-online":{"date-parts":[[2023,6,12]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.1145\/1060590.1060686"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1468-0262.2008.00846.x"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1176343654"},{"key":"e_1_3_2_1_4_1","volume-title":"Developing a COVID-19 mortality risk prediction model when individual-level data are not available. Nature communications 11, 1","author":"Barda Noam","year":"2020","unstructured":"Noam Barda , Dan Riesel , Amichay Akriv , Joseph Levy , Uriah Finkel , Gal Yona , Daniel Greenfeld , Shimon Sheiba , Jonathan Somer , Eitan Bachmat , 2020. Developing a COVID-19 mortality risk prediction model when individual-level data are not available. Nature communications 11, 1 ( 2020 ), 1\u20139. Noam Barda, Dan Riesel, Amichay Akriv, Joseph Levy, Uriah Finkel, Gal Yona, Daniel Greenfeld, Shimon Sheiba, Jonathan Somer, Eitan Bachmat, 2020. Developing a COVID-19 mortality risk prediction model when individual-level data are not available. Nature communications 11, 1 (2020), 1\u20139."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1093\/jamia\/ocaa283"},{"key":"e_1_3_2_1_6_1","article-title":"Algorithmic stability for adaptive data analysis","volume":"50","author":"Bassily Raef","year":"2021","unstructured":"Raef Bassily , Kobbi Nissim , Adam Smith , Thomas Steinke , Uri Stemmer , and Jonathan Ullman . 2021 . Algorithmic stability for adaptive data analysis . SIAM J. Comput. 50 , 3 (2021), STOC16\u2013377. Raef Bassily, Kobbi Nissim, Adam Smith, Thomas Steinke, Uri Stemmer, and Jonathan Ullman. 2021. Algorithmic stability for adaptive data analysis. SIAM J. Comput. 50, 3 (2021), STOC16\u2013377.","journal-title":"SIAM J. Comput."},{"key":"e_1_3_2_1_7_1","volume-title":"Practical Adversarial Multivalid Conformal Prediction. arXiv preprint arXiv:2206.01067","author":"Bastani Osbert","year":"2022","unstructured":"Osbert Bastani , Varun Gupta , Christopher Jung , Georgy Noarov , Ramya Ramalingam , and Aaron Roth . 2022. Practical Adversarial Multivalid Conformal Prediction. arXiv preprint arXiv:2206.01067 ( 2022 ). Osbert Bastani, Varun Gupta, Christopher Jung, Georgy Noarov, Ramya Ramalingam, and Aaron Roth. 2022. Practical Adversarial Multivalid Conformal Prediction. arXiv preprint arXiv:2206.01067 (2022)."},{"key":"e_1_3_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.1145\/3531146.3533149"},{"key":"e_1_3_2_1_9_1","volume-title":"Statistical modeling: The two cultures (with comments and a rejoinder by the author). Statistical science 16, 3","author":"Breiman Leo","year":"2001","unstructured":"Leo Breiman . 2001. Statistical modeling: The two cultures (with comments and a rejoinder by the author). Statistical science 16, 3 ( 2001 ), 199\u2013231. Leo Breiman. 2001. Statistical modeling: The two cultures (with comments and a rejoinder by the author). Statistical science 16, 3 (2001), 199\u2013231."},{"key":"e_1_3_2_1_10_1","volume-title":"Underspecification presents challenges for credibility in modern machine learning. arXiv preprint arXiv:2011.03395","author":"D\u2019Amour Alexander","year":"2020","unstructured":"Alexander D\u2019Amour , Katherine Heller , Dan Moldovan , Ben Adlam , Babak Alipanahi , Alex Beutel , Christina Chen , Jonathan Deaton , Jacob Eisenstein , Matthew D Hoffman , 2020. Underspecification presents challenges for credibility in modern machine learning. arXiv preprint arXiv:2011.03395 ( 2020 ). Alexander D\u2019Amour, Katherine Heller, Dan Moldovan, Ben Adlam, Babak Alipanahi, Alex Beutel, Christina Chen, Jonathan Deaton, Jacob Eisenstein, Matthew D Hoffman, 2020. Underspecification presents challenges for credibility in modern machine learning. arXiv preprint arXiv:2011.03395 (2020)."},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1080\/01621459.1982.10477856"},{"key":"e_1_3_2_1_12_1","doi-asserted-by":"publisher","DOI":"10.1214\/aos\/1176349736"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"crossref","first-page":"3445","DOI":"10.1007\/s11229-015-0953-4","article-title":"On individual risk","volume":"194","author":"Dawid Philip","year":"2017","unstructured":"Philip Dawid . 2017 . On individual risk . Synthese 194 , 9 (2017), 3445 \u2013 3474 . Philip Dawid. 2017. On individual risk. Synthese 194, 9 (2017), 3445\u20133474.","journal-title":"Synthese"},{"key":"e_1_3_2_1_15_1","first-page":"1","article-title":"The comparison and evaluation of forecasters","volume":"32","author":"DeGroot Morris H","year":"1983","unstructured":"Morris H DeGroot and Stephen E Fienberg . 1983 . The comparison and evaluation of forecasters . Journal of the Royal Statistical Society: Series D (The Statistician) 32 , 1 - 2 (1983), 12\u201322. Morris H DeGroot and Stephen E Fienberg. 1983. The comparison and evaluation of forecasters. Journal of the Royal Statistical Society: Series D (The Statistician) 32, 1-2 (1983), 12\u201322.","journal-title":"Journal of the Royal Statistical Society: Series D (The Statistician)"},{"key":"e_1_3_2_1_16_1","volume-title":"Generalization in adaptive data analysis and holdout reuse. Advances in Neural Information Processing Systems 28","author":"Dwork Cynthia","year":"2015","unstructured":"Cynthia Dwork , Vitaly Feldman , Moritz Hardt , Toni Pitassi , Omer Reingold , and Aaron Roth . 2015. Generalization in adaptive data analysis and holdout reuse. Advances in Neural Information Processing Systems 28 ( 2015 ). Cynthia Dwork, Vitaly Feldman, Moritz Hardt, Toni Pitassi, Omer Reingold, and Aaron Roth. 2015. Generalization in adaptive data analysis and holdout reuse. Advances in Neural Information Processing Systems 28 (2015)."},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/3406325.3451064"},{"volume-title":"Theory of cryptography conference","author":"Dwork Cynthia","key":"e_1_3_2_1_18_1","unstructured":"Cynthia Dwork , Frank McSherry , Kobbi Nissim , and Adam Smith . 2006. Calibrating noise to sensitivity in private data analysis . In Theory of cryptography conference . Springer , 265\u2013284. Cynthia Dwork, Frank McSherry, Kobbi Nissim, and Adam Smith. 2006. Calibrating noise to sensitivity in private data analysis. In Theory of cryptography conference. Springer, 265\u2013284."},{"key":"e_1_3_2_1_19_1","volume-title":"The algorithmic foundations of differential privacy. Foundations and Trends\u00ae in Theoretical Computer Science 9, 3\u20134","author":"Dwork Cynthia","year":"2014","unstructured":"Cynthia Dwork and Aaron Roth . 2014. The algorithmic foundations of differential privacy. Foundations and Trends\u00ae in Theoretical Computer Science 9, 3\u20134 ( 2014 ), 211\u2013407. Cynthia Dwork and Aaron Roth. 2014. The algorithmic foundations of differential privacy. Foundations and Trends\u00ae in Theoretical Computer Science 9, 3\u20134 (2014), 211\u2013407."},{"key":"e_1_3_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.1111\/j.1468-0262.2008.00847.x"},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1093\/biomet\/85.2.379"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1145\/3328526.3329624"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1016\/0022-0531(82)90099-0"},{"key":"e_1_3_2_1_24_1","volume-title":"An Algorithmic Framework for Bias Bounties. In 2022 ACM Conference on Fairness, Accountability, and Transparency","author":"Globus-Harris Ira","year":"2022","unstructured":"Ira Globus-Harris , Michael Kearns , and Aaron Roth . 2022 . An Algorithmic Framework for Bias Bounties. In 2022 ACM Conference on Fairness, Accountability, and Transparency ( Seoul, Republic of Korea) (FAccT \u201922). Association for Computing Machinery, New York, NY, USA, 1106\u20131124. https:\/\/doi.org\/10.1145\/3531146.3533172 10.1145\/3531146.3533172 Ira Globus-Harris, Michael Kearns, and Aaron Roth. 2022. An Algorithmic Framework for Bias Bounties. In 2022 ACM Conference on Fairness, Accountability, and Transparency (Seoul, Republic of Korea) (FAccT \u201922). Association for Computing Machinery, New York, NY, USA, 1106\u20131124. https:\/\/doi.org\/10.1145\/3531146.3533172"},{"key":"e_1_3_2_1_25_1","volume-title":"13th Innovations in Theoretical Computer Science Conference (ITCS","author":"Gupta Varun","year":"2022","unstructured":"Varun Gupta , Christopher Jung , Georgy Noarov , Mallesh M Pai , and Aaron Roth . 2022 . Online Multivalid Learning: Means, Moments, and Prediction Intervals . In 13th Innovations in Theoretical Computer Science Conference (ITCS 2022). Schloss Dagstuhl-Leibniz-Zentrum f\u00fcr Informatik. Varun Gupta, Christopher Jung, Georgy Noarov, Mallesh M Pai, and Aaron Roth. 2022. Online Multivalid Learning: Means, Moments, and Prediction Intervals. In 13th Innovations in Theoretical Computer Science Conference (ITCS 2022). Schloss Dagstuhl-Leibniz-Zentrum f\u00fcr Informatik."},{"key":"e_1_3_2_1_26_1","doi-asserted-by":"publisher","DOI":"10.1007\/s11229-006-9138-5"},{"key":"e_1_3_2_1_27_1","volume-title":"International Conference on Machine Learning. PMLR","author":"H\u00e9bert-Johnson Ursula","year":"2018","unstructured":"Ursula H\u00e9bert-Johnson , Michael Kim , Omer Reingold , and Guy Rothblum . 2018 . Multicalibration: Calibration for the (computationally-identifiable) masses . In International Conference on Machine Learning. PMLR , 1939\u20131948. Ursula H\u00e9bert-Johnson, Michael Kim, Omer Reingold, and Guy Rothblum. 2018. Multicalibration: Calibration for the (computationally-identifiable) masses. In International Conference on Machine Learning. PMLR, 1939\u20131948."},{"key":"e_1_3_2_1_28_1","volume-title":"Conference on Learning Theory. PMLR, 2634\u20132678","author":"Jung Christopher","year":"2021","unstructured":"Christopher Jung , Changhwa Lee , Mallesh Pai , Aaron Roth , and Rakesh Vohra . 2021 . Moment multicalibration for uncertainty estimation . In Conference on Learning Theory. PMLR, 2634\u20132678 . Christopher Jung, Changhwa Lee, Mallesh Pai, Aaron Roth, and Rakesh Vohra. 2021. Moment multicalibration for uncertainty estimation. In Conference on Learning Theory. PMLR, 2634\u20132678."},{"key":"e_1_3_2_1_29_1","volume-title":"Generalization Guarantees. In 11th Innovations in Theoretical Computer Science Conference (ITCS","author":"Jung Christopher","year":"2020","unstructured":"Christopher Jung , Katrina Ligett , Seth Neel , Aaron Roth , Saeed Sharifi-Malvajerdi , and Moshe Shenfeld . 2020 . A New Analysis of Differential Privacy\u2019s Generalization Guarantees. In 11th Innovations in Theoretical Computer Science Conference (ITCS 2020). Schloss Dagstuhl-Leibniz-Zentrum f\u00fcr Informatik. Christopher Jung, Katrina Ligett, Seth Neel, Aaron Roth, Saeed Sharifi-Malvajerdi, and Moshe Shenfeld. 2020. A New Analysis of Differential Privacy\u2019s Generalization Guarantees. In 11th Innovations in Theoretical Computer Science Conference (ITCS 2020). Schloss Dagstuhl-Leibniz-Zentrum f\u00fcr Informatik."},{"key":"e_1_3_2_1_30_1","volume-title":"International Conference on Machine Learning. PMLR, 6765\u20136774","author":"Marx Charles","year":"2020","unstructured":"Charles Marx , Flavio Calmon , and Berk Ustun . 2020 . Predictive multiplicity in classification . In International Conference on Machine Learning. PMLR, 6765\u20136774 . Charles Marx, Flavio Calmon, and Berk Ustun. 2020. Predictive multiplicity in classification. In International Conference on Machine Learning. PMLR, 6765\u20136774."},{"key":"e_1_3_2_1_31_1","volume-title":"Reconciling Individual Probability Forecasts. arXiv preprint arXiv:2209.01687","author":"Roth Aaron","year":"2022","unstructured":"Aaron Roth , Alexander Tolbert , and Scott Weinstein . 2022. Reconciling Individual Probability Forecasts. arXiv preprint arXiv:2209.01687 ( 2022 ). Aaron Roth, Alexander Tolbert, and Scott Weinstein. 2022. Reconciling Individual Probability Forecasts. arXiv preprint arXiv:2209.01687 (2022)."},{"key":"e_1_3_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.1007\/s001820300153"},{"key":"e_1_3_2_1_33_1","volume-title":"Calibration with many checking rules. Mathematics of operations Research 28, 1","author":"Sandroni Alvaro","year":"2003","unstructured":"Alvaro Sandroni , Rann Smorodinsky , and Rakesh V Vohra . 2003. Calibration with many checking rules. Mathematics of operations Research 28, 1 ( 2003 ), 141\u2013153. Alvaro Sandroni, Rann Smorodinsky, and Rakesh V Vohra. 2003. Calibration with many checking rules. Mathematics of operations Research 28, 1 (2003), 141\u2013153."}],"event":{"name":"FAccT '23: the 2023 ACM Conference on Fairness, Accountability, and Transparency","acronym":"FAccT '23","location":"Chicago IL USA"},"container-title":["2023 ACM Conference on Fairness, Accountability, and Transparency"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3593013.3593980","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3593013.3593980","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T16:48:02Z","timestamp":1750178882000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3593013.3593980"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2023,6,12]]},"references-count":32,"alternative-id":["10.1145\/3593013.3593980","10.1145\/3593013"],"URL":"https:\/\/doi.org\/10.1145\/3593013.3593980","relation":{},"subject":[],"published":{"date-parts":[[2023,6,12]]},"assertion":[{"value":"2023-06-12","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}