{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T18:54:03Z","timestamp":1782845643999,"version":"3.54.5"},"reference-count":71,"publisher":"Association for Computing Machinery (ACM)","issue":"FSE","license":[{"start":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T00:00:00Z","timestamp":1782777600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0\/legalcode"}],"funder":[{"DOI":"10.13039\/501100000038","name":"NSERC","doi-asserted-by":"crossref","award":["423227"],"award-info":[{"award-number":["423227"]}],"id":[{"id":"10.13039\/501100000038","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Proc. ACM Softw. Eng."],"published-print":{"date-parts":[[2026,6,30]]},"abstract":"<jats:p>\n                    The accelerating adoption of Large Language Models (LLMs) in software engineering (SE) has brought with it a silent crisis: unsustainable computational cost. While these models demonstrate remarkable capabilities in different SE tasks, they are unmanageably large, slow to deploy, memory-intensive, and carbon-heavy. This reality threatens not only the scalability and accessibility of AI-powered SE, but also its long-term environmental sustainability. The research challenge is clear: we must go beyond accuracy and address efficiency and environmental cost as first-class design constraints. To meet this challenge, we introduce Carbon-Taxed Transformers (CTT), a systematic multi-architectural compression principled pipeline ordering inspired by economic carbon taxation principles. Drawing from the economic concept of carbon pricing, CTT operationalizes a computational carbon tax that penalizes architectural inefficiencies and rewards deployment-ready compression. We evaluate CTT across three core SE tasks: code clone detection, code summarization, and code generation, with models spanning encoder-only, encoder-decoder, and decoder-only architecture. Our results show that CTT delivers on inference: (1) up to 49\u00d7 memory reduction, (2) time reduction up to 8-10\u00d7 for clone detection, up to 3\u00d7 for summarization, and 4\u20137\u00d7 for generation, (3) up to 81% reduction in CO\n                    <jats:sub>2<\/jats:sub>\n                    emissions and (4) CTT retains around 98% accuracy on clone detection, around 89% on summarization, and up to 91% (textual metrics) and 68% (pass@1) for generation. Two ablation studies show that pipeline ordering and individual component contributions are both essential, providing empirical justification for CTT\u2019s design and effectiveness. This work establishes a viable path toward responsible AI in SE through aggressive yet performance-preserving compression.\n                  <\/jats:p>","DOI":"10.1145\/3797075","type":"journal-article","created":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T17:06:14Z","timestamp":1782839174000},"page":"1035-1058","source":"Crossref","is-referenced-by-count":0,"title":["Carbon-Taxed Transformers: A Green Compression Pipeline for Overgrown Language Models"],"prefix":"10.1145","volume":"3","author":[{"ORCID":"https:\/\/orcid.org\/0009-0006-1180-2369","authenticated-orcid":false,"given":"Ajmain Inqiad","family":"Alam","sequence":"first","affiliation":[{"name":"University of Saskatchewan, Department of Computer Science, Saskatoon, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9470-4233","authenticated-orcid":false,"given":"Palash","family":"Roy","sequence":"additional","affiliation":[{"name":"University of Saskatchewan, Department of Computer Science, Saskatoon, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-0519-6164","authenticated-orcid":false,"given":"Chanchal K.","family":"Roy","sequence":"additional","affiliation":[{"name":"University of Saskatchewan, Department of Computer Science, Saskatoon, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1247-7781","authenticated-orcid":false,"given":"Banani","family":"Roy","sequence":"additional","affiliation":[{"name":"University of Saskatchewan, Department of Computer Science, Saskatoon, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"ORCID":"https:\/\/orcid.org\/0000-0003-1113-1754","authenticated-orcid":false,"given":"Kevin A.","family":"Schneider","sequence":"additional","affiliation":[{"name":"University of Saskatchewan, Department of Computer Science, Saskatoon, Canada"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2026,6,30]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.211"},{"key":"e_1_2_1_2_1","doi-asserted-by":"publisher","unstructured":"Jacob Austin Augustus Odena Maxwell Nye Maarten Bosma Henryk Michalewski David Dohan Ellen Jiang Carrie Cai Michael Terry Quoc Le et al. 2021. Program Synthesis with Large Language Models. arXiv preprint arXiv:2108.07732 (2021). doi:10.48550\/arXiv.2108.07732 10.48550\/arXiv.2108.07732","DOI":"10.48550\/arXiv.2108.07732"},{"key":"e_1_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2212.05901"},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-short.1"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1908.09791"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","unstructured":"Yupeng Chang Xu Wang Jindong Wang Yuan Wu Linyi Yang Kaijie Zhu Hao Chen Xiaoyuan Yi Cunxiang Wang Yidong Wang Wei Ye Yue Zhang Yi Chang Philip S. Yu Qiang Yang and Xing Xie. 2024. A Survey on Evaluation of Large Language Models. ACM transactions on intelligent systems and technology 15 3 1-45. doi:10.1145\/3641289 10.1145\/3641289","DOI":"10.1145\/3641289"},{"key":"e_1_2_1_7_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2020\/341"},{"key":"e_1_2_1_8_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2107.03374"},{"key":"e_1_2_1_9_1","doi-asserted-by":"publisher","unstructured":"Benoit Courty Victor Schmidt Sasha Luccioni Goyal-Kamal MarionCoutarel Boris Feld et al. 2024. mlco2\/codecarbon: v2.4.1. doi:10.5281\/zenodo.11171501 10.5281\/zenodo.11171501","DOI":"10.5281\/zenodo.11171501"},{"key":"e_1_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.52202\/068431-2198"},{"key":"e_1_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1808.05377"},{"key":"e_1_2_1_12_1","unstructured":"Sol Farahmand. 2025. Working with LLMS: Using Lora vs quantization vs both. https:\/\/medium.com\/@sol. farahmand1986\/working-with-llms-using-lora-vs-quantization-vs-both-8b20c7db427d"},{"key":"e_1_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.139"},{"key":"e_1_2_1_14_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1803.03635"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2210.17323"},{"key":"e_1_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/"},{"key":"e_1_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2022.acl-long.499"},{"key":"e_1_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/3695988"},{"key":"e_1_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1902.00751"},{"key":"e_1_2_1_20_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2106.09685"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","unstructured":"Binyuan Hui Jian Yang Zeyu Cui Jiaxi Yang Dayiheng Liu Lei Zhang Tianyu Liu Jiajun Zhang Bowen Yu Kai Dang et al. 2024. Qwen2. 5-Coder Technical Report. arXiv preprint arXiv:2409.12186. doi:10.48550\/arXiv.2409.12186 10.48550\/arXiv.2409.12186","DOI":"10.48550\/arXiv.2409.12186"},{"key":"e_1_2_1_22_1","doi-asserted-by":"publisher","unstructured":"Hamel Husain Ho-Hsiang Wu Tiferet Gazit Miltiadis Allamanis and Marc Brockschmidt. 2019. Codesearchnet challenge: Evaluating the state of semantic code search. arXiv preprint arXiv:1909.09436. doi:10.48550\/arXiv.1909.09436 10.48550\/arXiv.1909.09436","DOI":"10.48550\/arXiv.1909.09436"},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2018.00286"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.372"},{"key":"e_1_2_1_25_1","doi-asserted-by":"publisher","DOI":"10.1145\/3534678.3539260"},{"key":"e_1_2_1_26_1","doi-asserted-by":"publisher","unstructured":"Baolin Li Yankai Jiang Vijay Gadepally and Devesh Tiwari. 2024. Toward sustainable genai using generation directives for carbon-friendly large language model inference. arXiv preprint arXiv:2403.12900. doi:10.48550\/arXiv.2403.12900 10.48550\/arXiv.2403.12900","DOI":"10.48550\/arXiv.2403.12900"},{"key":"e_1_2_1_27_1","doi-asserted-by":"publisher","unstructured":"Raymond Li Loubna Ben allal Yangtian Zi Niklas Muennighoff Denis Kocetkov Chenghao Mou Marc Marone Christopher Akiki Jia LI Jenny Chim Qian Liu Evgenii Zheltonozhskii Terry Yue Zhuo Thomas Wang Olivier Dehaene Joel Lamy-Poirier Joao Monteiro Nicolas Gontier Ming-Ho Yee Logesh Kumar Umapathi Jian Zhu Ben Lipkin Muhtasham Oblokulov Zhiruo Wang Rudra Murthy Jason T Stillerman Siva Sankalp Patel Dmitry Abulkhanov Marco Zocca Manan Dey Zhihan Zhang Urvashi Bhattacharyya Wenhao Yu Sasha Luccioni Paulo Villegas Fedor Zhdanov Tony Lee Nadav Timor Jennifer Ding Claire S Schlesinger Hailey Schoelkopf Jan Ebert Tri Dao Mayank Mishra Alex Gu Carolyn Jane Anderson Brendan Dolan-Gavitt Danish Contractor Siva Reddy Daniel Fried Dzmitry Bahdanau Yacine Jernite Carlos Mu\u00f1oz Ferrandis Sean Hughes Thomas Wolf Arjun Guha Leandro Von Werra and Harm de Vries. 2023. StarCoder: may the source be with you! Transactions on Machine Learning Research. doi:10.48550\/arXiv.2305.06161 10.48550\/arXiv.2305.06161","DOI":"10.48550\/arXiv.2305.06161"},{"key":"e_1_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.acllong.353"},{"key":"e_1_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1126\/science.abq1158"},{"key":"e_1_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2007.10319"},{"key":"e_1_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2306.00978"},{"key":"e_1_2_1_32_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1907.11692"},{"key":"e_1_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1145\/3630106.3658542"},{"key":"e_1_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2106.04647"},{"key":"e_1_2_1_35_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1905.10650"},{"key":"e_1_2_1_36_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2203.13474"},{"key":"e_1_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.2022.3148714"},{"key":"e_1_2_1_38_1","doi-asserted-by":"publisher","unstructured":"David Patterson Joseph Gonzalez Quoc Le Chen Liang Lluis-Miquel Munguia Daniel Rothchild David So Maud Texier and Jeff Dean. 2021. Carbon emissions and large neural network training. arXiv preprint arXiv:2104.10350. doi:10.48550\/arXiv.2104.10350 10.48550\/arXiv.2104.10350","DOI":"10.48550\/arXiv.2104.10350"},{"key":"e_1_2_1_39_1","doi-asserted-by":"publisher","unstructured":"Sida Peng Eirini Kalliamvakou Peter Cihon and Mert Demirer. 2023. The impact of ai on developer productivity: Evidence from github copilot. arXiv preprint arXiv:2302.06590. doi:10.48550\/arXiv.2302.06590 10.48550\/arXiv.2302.06590","DOI":"10.48550\/arXiv.2302.06590"},{"key":"e_1_2_1_40_1","unstructured":"Salesforce Research. 2023. Discussion on CodeT5+ model execution error. https:\/\/huggingface.co\/Salesforce\/codet5p- 2b\/discussions\/9. Accessed: 2025-09-09."},{"key":"e_1_2_1_41_1","unstructured":"Salesforce Research. 2023. GitHub Issue: AssertionError when running CodeT5+ models. https:\/\/github.com\/salesforce\/ CodeT5\/issues\/192. Accessed: 2025-09-09."},{"key":"e_1_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-642-04898-2_616"},{"key":"e_1_2_1_43_1","doi-asserted-by":"publisher","DOI":"10.1145\/3715773"},{"key":"e_1_2_1_44_1","doi-asserted-by":"publisher","unstructured":"Victor Sanh Lysandre Debut Julien Chaumond and Thomas Wolf. 2019. DistilBERT a distilled version of BERT: smaller faster cheaper and lighter. arXiv preprint arXiv:1910.01108. doi:10.48550\/arXiv.1910.01108 10.48550\/arXiv.1910.01108","DOI":"10.48550\/arXiv.1910.01108"},{"key":"e_1_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2005.07683"},{"key":"e_1_2_1_46_1","volume-title":"OpenAI CEO claims saying \"please\" and \"thank you\" to CHATGPT costs \"tens of millions of dollars\" -here's why. People.com (Apr","author":"Sheffield Toria","year":"2025","unstructured":"Toria Sheffield. 2025. OpenAI CEO claims saying \"please\" and \"thank you\" to CHATGPT costs \"tens of millions of dollars\" -here's why. People.com (Apr 2025). https:\/\/people.com\/open-ai-ceo-claims-saying-please-thank-you-tochatgpt-costs-millions-of-dollars-in-electricity-bills-11721523"},{"key":"e_1_2_1_47_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i05.6409"},{"key":"e_1_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2303.06865"},{"key":"e_1_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1145\/3639475.3640097"},{"key":"e_1_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1145\/3551349.3556964"},{"key":"e_1_2_1_51_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.2109.08668"},{"key":"e_1_2_1_52_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai"},{"key":"e_1_2_1_53_1","doi-asserted-by":"publisher","DOI":"10.2307\/2331554"},{"key":"e_1_2_1_54_1","doi-asserted-by":"publisher","DOI":"10.1007\/s10515-024-00421-4"},{"key":"e_1_2_1_55_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-981-16-1927-4_7"},{"key":"e_1_2_1_56_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1905.11946"},{"key":"e_1_2_1_57_1","doi-asserted-by":"publisher","DOI":"10.48550\/arXiv.1903.12136"},{"key":"e_1_2_1_58_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P19-1580"},{"key":"e_1_2_1_59_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2024.3368208"},{"key":"e_1_2_1_60_1","doi-asserted-by":"publisher","unstructured":"Wenhui Wang Furu Wei Li Dong Hangbo Bao Nan Yang and Ming Zhou. 2020. Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers. Advances in neural information processing systems 33 5776-5788. doi:10.48550\/arXiv.2002.10957 10.48550\/arXiv.2002.10957","DOI":"10.48550\/arXiv.2002.10957"},{"key":"e_1_2_1_61_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2023.emnlp-main.68"},{"key":"e_1_2_1_62_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlpmain.685"},{"key":"e_1_2_1_63_1","doi-asserted-by":"publisher","DOI":"10.1145\/3611643.3616302"},{"key":"e_1_2_1_64_1","doi-asserted-by":"publisher","DOI":"10.3390\/app122111220"},{"key":"e_1_2_1_65_1","doi-asserted-by":"publisher","DOI":"10.1145\/3520312.3534862"},{"key":"e_1_2_1_66_1","doi-asserted-by":"publisher","DOI":"10.1145\/3447548.3467262"},{"key":"e_1_2_1_67_1","doi-asserted-by":"publisher","unstructured":"An Yang Baosong Yang Binyuan Hui Bo Zheng Bowen Yu Chang Zhou et al. 2024. Qwen2 Technical Report. arXiv preprint arXiv:2407.10671. doi:10.48550\/arXiv.2407.10671 10.48550\/arXiv.2407.10671","DOI":"10.48550\/arXiv.2407.10671"},{"key":"e_1_2_1_68_1","doi-asserted-by":"publisher","DOI":"10.1145\/3705725"},{"key":"e_1_2_1_69_1","doi-asserted-by":"publisher","DOI":"10.1109\/EMC2-NIPS53020.2019.00016"},{"key":"e_1_2_1_70_1","first-page":"848","article-title":"DietCode: Automatic optimization for dynamic tensor programs","volume":"4","author":"Zheng Bojian","year":"2022","unstructured":"Bojian Zheng, Ziheng Jiang, Cody Hao Yu, Haichen Shen, Joshua Fromm, Yizhi Liu, Yida Wang, Luis Ceze, Tianqi Chen, and Gennady Pekhimenko. 2022. DietCode: Automatic optimization for dynamic tensor programs. Proceedings of Machine Learning and Systems 4, 848-863.","journal-title":"Proceedings of Machine Learning and Systems"},{"key":"e_1_2_1_71_1","doi-asserted-by":"publisher","unstructured":"Shuchang Zhou Zekun Ni Xinyu Zhou He Wen Yuxin Wu and Yuheng Zou. 2016. DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients. ArXiv abs\/1606.06160. doi:10.48550\/arXiv.1606.06160 10.48550\/arXiv.1606.06160","DOI":"10.48550\/arXiv.1606.06160"}],"container-title":["Proceedings of the ACM on Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3797075","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2026,6,30]],"date-time":"2026-06-30T17:51:56Z","timestamp":1782841916000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3797075"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2026,6,30]]},"references-count":71,"journal-issue":{"issue":"FSE","published-print":{"date-parts":[[2026,6,30]]}},"alternative-id":["10.1145\/3797075"],"URL":"https:\/\/doi.org\/10.1145\/3797075","relation":{},"ISSN":["2994-970X"],"issn-type":[{"value":"2994-970X","type":"electronic"}],"subject":[],"published":{"date-parts":[[2026,6,30]]}}}