{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,8]],"date-time":"2026-07-08T12:22:16Z","timestamp":1783513336003,"version":"3.55.0"},"publisher-location":"New York, NY, USA","reference-count":34,"publisher":"ACM","license":[{"start":{"date-parts":[[2026,4,12]],"date-time":"2026-04-12T00:00:00Z","timestamp":1775952000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2026,4,12]]},"DOI":"10.1145\/3786582.3786845","type":"proceedings-article","created":{"date-parts":[[2026,7,8]],"date-time":"2026-07-08T11:52:47Z","timestamp":1783511567000},"page":"231-235","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":0,"title":["When to Answer and When to Defer: A Decision Framework for Reliable Code Predictions"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0009-0005-6129-2865","authenticated-orcid":false,"given":"Ravishka","family":"Rathnasuriya","sequence":"first","affiliation":[{"name":"University of Texas at Dallas, Richardson, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5338-7347","authenticated-orcid":false,"given":"Wei","family":"Yang","sequence":"additional","affiliation":[{"name":"University of Texas at Dallas, Richardson, USA"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,7,8]]},"reference":[{"key":"e_1_3_3_1_2_2","doi-asserted-by":"crossref","unstructured":"Yihong Dong Jiazheng Ding Xue Jiang Ge Li Zhuo Li and Zhi Jin. 2025. Codescore: Evaluating code generation by learning code execution. ACM Transactions on Software Engineering and Methodology 34 3 (2025) 1\u201322.","DOI":"10.1145\/3695991"},{"key":"e_1_3_3_1_3_2","first-page":"1050","volume-title":"international conference on machine learning","author":"Gal Yarin","year":"2016","unstructured":"Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning. PMLR, 1050\u20131059."},{"key":"e_1_3_3_1_4_2","unstructured":"Chuan Guo Geoff Pleiss Yu Sun and Kilian\u00a0Q. Weinberger. 2017. On Calibration of Modern Neural Networks. arxiv:https:\/\/arXiv.org\/abs\/1706.04599\u00a0[cs.LG]"},{"key":"e_1_3_3_1_5_2","unstructured":"Daya Guo Qihao Zhu Dejian Yang Zhenda Xie Kai Dong Wentao Zhang Guanting Chen Xiao Bi Yu Wu YK Li et\u00a0al. 2024. DeepSeek-Coder: When the Large Language Model Meets Programming\u2013The Rise of Code Intelligence. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2401.14196 (2024)."},{"key":"e_1_3_3_1_6_2","unstructured":"Kartik Gupta Amir Rahimi Thalaiyasingam Ajanthan Thomas Mensink Cristian Sminchisescu and Richard Hartley. 2020. Calibration of neural networks using splines. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2006.12800 (2020)."},{"key":"e_1_3_3_1_7_2","unstructured":"Dan Hendrycks and Kevin Gimpel. 2018. A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networks. arxiv:https:\/\/arXiv.org\/abs\/1610.02136\u00a0[cs.NE]"},{"key":"e_1_3_3_1_8_2","volume-title":"International Conference on Software Engineering (ICSE): New Ideas and Emerging Results (NIER)","author":"Hu Qiang","year":"2023","unstructured":"Qiang Hu, Yuejun Guo, Xiaofei Xie, Maxime Cordy, Mike Papadakis, Lei Ma, and Yves Le\u00a0Traon. 2023. CodeS: towards code model generalization under distribution shift. In International Conference on Software Engineering (ICSE): New Ideas and Emerging Results (NIER)."},{"key":"e_1_3_3_1_9_2","unstructured":"Binyuan Hui Jian Yang Zeyu Cui Jiaxi Yang Dayiheng Liu Lei Zhang Tianyu Liu Jiajun Zhang Bowen Yu Keming Lu et\u00a0al. 2024. Qwen2. 5-coder technical report. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2409.12186 (2024)."},{"key":"e_1_3_3_1_10_2","doi-asserted-by":"crossref","unstructured":"Xue Jiang Yihong Dong Lecheng Wang Zheng Fang Qiwei Shang Ge Li Zhi Jin and Wenpin Jiao. 2024. Self-planning code generation with large language models. ACM Transactions on Software Engineering and Methodology 33 7 (2024) 1\u201330.","DOI":"10.1145\/3672456"},{"key":"e_1_3_3_1_11_2","unstructured":"Balaji Lakshminarayanan Alexander Pritzel and Charles Blundell. 2017. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems 30 (2017)."},{"key":"e_1_3_3_1_12_2","unstructured":"Raymond Li Loubna\u00a0Ben Allal Yangtian Zi Niklas Muennighoff Denis Kocetkov Chenghao Mou Marc Marone Christopher Akiki Jia Li Jenny Chim et\u00a0al. 2023. Starcoder: may the source be with you! arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2305.06161 (2023)."},{"key":"e_1_3_3_1_13_2","unstructured":"Yufei Li Simin Chen and Wei Yang. 2021. Estimating predictive uncertainty under program data distribution shift. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2107.10989 (2021)."},{"key":"e_1_3_3_1_14_2","doi-asserted-by":"publisher","DOI":"10.52202\/075280-0943"},{"key":"e_1_3_3_1_15_2","volume-title":"First Conference on Language Modeling","author":"Liu Jiawei","year":"2024","unstructured":"Jiawei Liu, Songrun Xie, Junhao Wang, Yuxiang Wei, Yifeng Ding, and Lingming Zhang. 2024. Evaluating Language Models for Efficient Code Generation. In First Conference on Language Modeling. https:\/\/openreview.net\/forum?id=IBCBMeAhmC"},{"key":"e_1_3_3_1_16_2","unstructured":"Shuai Lu Daya Guo Shuo Ren Junjie Huang Alexey Svyatkovskiy Ambrosio Blanco Colin Clement Dawn Drain Daxin Jiang Duyu Tang et\u00a0al. 2021. Codexglue: A machine learning benchmark dataset for code understanding and generation. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2102.04664 (2021)."},{"key":"e_1_3_3_1_17_2","volume-title":"Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI","author":"Monarch Robert\u00a0Munro","year":"2021","unstructured":"Robert\u00a0Munro Monarch. 2021. Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI. Simon and Schuster."},{"key":"e_1_3_3_1_18_2","doi-asserted-by":"publisher","DOI":"10.1109\/SP.2018.00056"},{"key":"e_1_3_3_1_19_2","doi-asserted-by":"publisher","DOI":"10.1109\/IESYS.2017.8233558"},{"key":"e_1_3_3_1_20_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-Companion66252.2025.00041"},{"key":"e_1_3_3_1_21_2","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE-Companion66252.2025.00073"},{"key":"e_1_3_3_1_22_2","first-page":"676","volume-title":"2025 IEEE\/ACM 47th International Conference on Software Engineering (ICSE)","author":"Rathnasuriya Ravishka","year":"2025","unstructured":"Ravishka Rathnasuriya, Zijie Zhao, and Wei Yang. 2025. CodeImprove: Program Adaptation for Deep Code Models. In 2025 IEEE\/ACM 47th International Conference on Software Engineering (ICSE). IEEE Computer Society, 676\u2013676."},{"key":"e_1_3_3_1_23_2","unstructured":"Baptiste Roziere Jonas Gehring Fabian Gloeckle Sten Sootla Itai Gat Xiaoqing\u00a0Ellen Tan Yossi Adi Jingyu Liu Romain Sauvestre Tal Remez et\u00a0al. 2023. Code llama: Open foundation models for code. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2308.12950 (2023)."},{"key":"e_1_3_3_1_24_2","unstructured":"Anjana Sarkar and Soumyendu Sarkar. 2025. Survey of LLM Agent Communication with MCP: A Software Design Pattern Centric Review. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2506.05364 (2025)."},{"key":"e_1_3_3_1_25_2","doi-asserted-by":"crossref","unstructured":"Claude\u00a0Elwood Shannon. 1948. A mathematical theory of communication. The Bell system technical journal 27 3 (1948) 379\u2013423.","DOI":"10.1002\/j.1538-7305.1948.tb01338.x"},{"key":"e_1_3_3_1_26_2","unstructured":"Claudio Spiess David Gros Kunal\u00a0Suresh Pai Michael Pradel Md\u00a0Rafiqul\u00a0Islam Rabin Amin Alipour Susmit Jha Prem Devanbu and Toufique Ahmed. 2024. Calibration and correctness of language models for code. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2402.02047 (2024)."},{"key":"e_1_3_3_1_27_2","unstructured":"Jacob Steinhardt and Percy\u00a0S Liang. 2016. Unsupervised risk estimation using only conditional independence structure. Advances in Neural Information Processing Systems 29 (2016)."},{"key":"e_1_3_3_1_28_2","doi-asserted-by":"crossref","unstructured":"Zhao Tian Junjie Chen and Xiangyu Zhang. 2023. On-the-fly Improving Performance of Deep Code Models via Input Denoising. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2308.09969 (2023).","DOI":"10.1109\/ASE56229.2023.00166"},{"key":"e_1_3_3_1_29_2","doi-asserted-by":"crossref","first-page":"824","DOI":"10.1145\/3377811.3380343","volume-title":"Proceedings of the ACM\/IEEE 42nd International Conference on Software Engineering","author":"Tonder Rijnard van","year":"2020","unstructured":"Rijnard van Tonder and Claire\u00a0Le Goues. 2020. Tailoring programs for static analysis via program transformation. In Proceedings of the ACM\/IEEE 42nd International Conference on Software Engineering. 824\u2013834."},{"key":"e_1_3_3_1_30_2","unstructured":"Roman Vashurin Ekaterina Fadeeva Artem Vazhentsev Lyudmila Rvanova Akim Tsvigun Daniil Vasilev Rui Xing Abdelrahman\u00a0Boda Sadallah Kirill Grishchenkov Sergey Petrakov et\u00a0al. 2024. Benchmarking uncertainty quantification methods for large language models with lm-polygraph. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2406.15627 (2024)."},{"key":"e_1_3_3_1_31_2","unstructured":"Ruslan Vasilev and Alexander D\u2019yakonov. 2023. Calibration of neural networks. arXiv preprint arXiv:https:\/\/arXiv.org\/abs\/2303.10761 (2023)."},{"key":"e_1_3_3_1_32_2","doi-asserted-by":"publisher","DOI":"10.1145\/3510003.3510146"},{"key":"e_1_3_3_1_33_2","doi-asserted-by":"crossref","unstructured":"Noam Yefet Uri Alon and Eran Yahav. 2020. Adversarial examples for models of code. Proceedings of the ACM on Programming Languages 4 OOPSLA (2020) 1\u201330.","DOI":"10.1145\/3428230"},{"key":"e_1_3_3_1_34_2","doi-asserted-by":"publisher","unstructured":"Weiwei Zhang Shengjian Guo Hongyu Zhang Yulei Sui Yinxing Xue and Yun Xu. 2023. Challenging Machine Learning-based Clone Detectors via Semantic-preserving Code Transformations. IEEE Transactions on Software Engineering 49 5 (May 2023) 3052\u20133070. 10.1109\/TSE.2023.3240118 arxiv:https:\/\/arXiv.org\/abs\/2111.10793\u00a0[cs]","DOI":"10.1109\/TSE.2023.3240118"},{"key":"e_1_3_3_1_35_2","first-page":"1","volume-title":"Proceedings of the IEEE\/ACM 46th international conference on software engineering","author":"Zhou Zhenhao","year":"2024","unstructured":"Zhenhao Zhou, Chaofeng Sha, and Xin Peng. 2024. On calibration of pre-trained code models. In Proceedings of the IEEE\/ACM 46th international conference on software engineering. 1\u201313."}],"event":{"name":"ICSE-NIER '26: 2026 IEEE\/ACM 48th International Conference on Software Engineering","location":"Rio de Janeiro Brazil","acronym":"ICSE-NIER '26","sponsor":["SIGSOFT ACM Special Interest Group on Software Engineering"]},"container-title":["Proceedings of the IEEE\/ACM 48th International Conference on Software Engineering"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3786582.3786845","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,7,8]],"date-time":"2026-07-08T11:55:00Z","timestamp":1783511700000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3786582.3786845"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,4,12]]},"references-count":34,"alternative-id":["10.1145\/3786582.3786845","10.1145\/3786582"],"URL":"https:\/\/doi.org\/10.1145\/3786582.3786845","relation":{},"subject":[],"published":{"date-parts":[[2026,4,12]]},"assertion":[{"value":"2026-07-08","order":3,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}