{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,7,3]],"date-time":"2026-07-03T16:40:46Z","timestamp":1783096846394,"version":"3.54.6"},"publisher-location":"New York, NY, USA","reference-count":50,"publisher":"ACM","license":[{"start":{"date-parts":[[2022,11,7]],"date-time":"2022-11-07T00:00:00Z","timestamp":1667779200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"funder":[{"DOI":"10.13039\/100000001","name":"NSF (National Science Foundation)","doi-asserted-by":"publisher","award":["62102244"],"award-info":[{"award-number":["62102244"]}],"id":[{"id":"10.13039\/100000001","id-type":"DOI","asserted-by":"publisher"}]},{"name":"CCF-Baidu Open Fund","award":["2021PP15002000"],"award-info":[{"award-number":["2021PP15002000"]}]},{"name":"CCF-Tencent Rhino-Bird Young Faculty Open Research Fund","award":[""],"award-info":[{"award-number":[""]}]}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2022,11,7]]},"DOI":"10.1145\/3540250.3549094","type":"proceedings-article","created":{"date-parts":[[2022,11,9]],"date-time":"2022-11-09T20:46:22Z","timestamp":1668026782000},"page":"1073-1084","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":37,"title":["Diet code is healthy: simplifying programs for pre-trained models of code"],"prefix":"10.1145","author":[{"given":"Zhaowei","family":"Zhang","sequence":"first","affiliation":[{"name":"Shanghai Jiao Tong University, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Hongyu","family":"Zhang","sequence":"additional","affiliation":[{"name":"University of Newcastle, Australia"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Beijun","family":"Shen","sequence":"additional","affiliation":[{"name":"Shanghai Jiao Tong University, China"}],"role":[{"vocabulary":"crossref","role":"author"}]},{"given":"Xiaodong","family":"Gu","sequence":"additional","affiliation":[{"name":"Shanghai Jiao Tong University, China"}],"role":[{"vocabulary":"crossref","role":"author"}]}],"member":"320","published-online":{"date-parts":[[2022,11,9]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.211"},{"key":"e_1_3_2_1_2_1","volume-title":"International Conference on Learning Representations.","author":"Allamanis Miltiadis","year":"2018","unstructured":"Miltiadis Allamanis , Marc Brockschmidt , and Mahmoud Khademi . 2018 . Learning to Represent Programs with Graphs . In International Conference on Learning Representations. Miltiadis Allamanis, Marc Brockschmidt, and Mahmoud Khademi. 2018. Learning to Represent Programs with Graphs. In International Conference on Learning Representations."},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3290353"},{"key":"e_1_3_2_1_4_1","volume-title":"Proceedings of the 3rd International Conference on Learning Representations (ICLR).","author":"Bahdanau Dzmitry","year":"2015","unstructured":"Dzmitry Bahdanau , Kyung Hyun Cho , and Yoshua Bengio . 2015 . Neural machine translation by jointly learning to align and translate . In Proceedings of the 3rd International Conference on Learning Representations (ICLR). Dzmitry Bahdanau, Kyung Hyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the 3rd International Conference on Learning Representations (ICLR)."},{"key":"e_1_3_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE.2019.00014"},{"key":"e_1_3_2_1_6_1","volume-title":"Proceedings of the 42nd Annual Meeting of the Cognitive Science Society.","author":"Casalnuovo Casey","year":"2020","unstructured":"Casey Casalnuovo , E Morgan , and P Devanbu . 2020 . Does surprisal predict code comprehension difficulty . In Proceedings of the 42nd Annual Meeting of the Cognitive Science Society. Casey Casalnuovo, E Morgan, and P Devanbu. 2020. Does surprisal predict code comprehension difficulty. In Proceedings of the 42nd Annual Meeting of the Cognitive Science Society."},{"key":"e_1_3_2_1_7_1","volume-title":"Proceedings of the 30th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC\/FSE).","author":"Chakraborty Saikat","year":"2020","unstructured":"Saikat Chakraborty , Toufique Ahmed , Yangruibo Ding , Premkumar Devanbu , and Baishakhi Ray . 2020 . NatGen: Generative pre-training by\" Naturalizing\" source code . In Proceedings of the 30th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC\/FSE). Saikat Chakraborty, Toufique Ahmed, Yangruibo Ding, Premkumar Devanbu, and Baishakhi Ray. 2020. NatGen: Generative pre-training by\" Naturalizing\" source code. In Proceedings of the 30th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC\/FSE)."},{"key":"e_1_3_2_1_8_1","unstructured":"Yu Cheng Duo Wang Pan Zhou and Tao Zhang. 2017. A survey of model compression and acceleration for deep neural networks. arXiv preprint arXiv:1710.09282. \t\t\t\t  Yu Cheng Duo Wang Pan Zhou and Tao Zhang. 2017. A survey of model compression and acceleration for deep neural networks. arXiv preprint arXiv:1710.09282."},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSP.2017.2765695"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/D14-1179"},{"key":"e_1_3_2_1_11_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSR52588.2021.00024"},{"key":"e_1_3_2_1_12_1","volume-title":"International Conference on Learning Representations.","author":"Clark Kevin","year":"2019","unstructured":"Kevin Clark , Minh-Thang Luong , Quoc V Le , and Christopher D Manning . 2019 . ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators . In International Conference on Learning Representations. Kevin Clark, Minh-Thang Luong, Quoc V Le, and Christopher D Manning. 2019. ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators. In International Conference on Learning Representations."},{"key":"e_1_3_2_1_13_1","volume-title":"Cross-lingual language model pretraining. Advances in neural information processing systems, 32","author":"Conneau Alexis","year":"2019","unstructured":"Alexis Conneau and Guillaume Lample . 2019. Cross-lingual language model pretraining. Advances in neural information processing systems, 32 ( 2019 ). Alexis Conneau and Guillaume Lample. 2019. Cross-lingual language model pretraining. Advances in neural information processing systems, 32 (2019)."},{"key":"e_1_3_2_1_14_1","volume-title":"Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies","volume":"1","author":"Devlin Jacob","year":"2019","unstructured":"Jacob Devlin , Ming-Wei Chang , Kenton Lee , and Kristina Toutanova . 2019 . BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding . In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , Volume 1 (Long and Short Papers). 4171\u20134186. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 4171\u20134186."},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.139"},{"key":"e_1_3_2_1_16_1","volume-title":"The Java language specification","author":"Gosling James","unstructured":"James Gosling , Bill Joy , Guy Steele , and Gilad Bracha . 2000. The Java language specification . Addison-Wesley Professional . James Gosling, Bill Joy, Guy Steele, and Gilad Bracha. 2000. The Java language specification. Addison-Wesley Professional."},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1810295.1810335"},{"key":"e_1_3_2_1_18_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2012.6227135"},{"key":"e_1_3_2_1_19_1","volume-title":"Floating point operations in matrix-vector calculus","author":"Hunger Raphael","unstructured":"Raphael Hunger . 2005. Floating point operations in matrix-vector calculus . Munich University of Technology , Inst . for Circuit Theory and Signal. Raphael Hunger. 2005. Floating point operations in matrix-vector calculus. Munich University of Technology, Inst. for Circuit Theory and Signal."},{"key":"e_1_3_2_1_20_1","unstructured":"Hamel Husain Ho-Hsiang Wu Tiferet Gazit Miltiadis Allamanis and Marc Brockschmidt. 2019. Codesearchnet challenge: evaluating the state of semantic code search. arXiv preprint arXiv:1909.09436. \t\t\t\t  Hamel Husain Ho-Hsiang Wu Tiferet Gazit Miltiadis Allamanis and Marc Brockschmidt. 2019. Codesearchnet challenge: evaluating the state of semantic code search. arXiv preprint arXiv:1909.09436."},{"key":"e_1_3_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P15-1162"},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.5555\/3155562.3155583"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2002.1019480"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/ASE51524.2021.9678927"},{"key":"e_1_3_2_1_25_1","doi-asserted-by":"crossref","unstructured":"Sehoon Kim Sheng Shen David Thorsley Amir Gholami Woosuk Kwon Joseph Hassoun and Kurt Keutzer. 2021. Learned token pruning for transformers. arXiv preprint arXiv:2107.00910. \t\t\t\t  Sehoon Kim Sheng Shen David Thorsley Amir Gholami Woosuk Kwon Joseph Hassoun and Kurt Keutzer. 2021. Learned token pruning for transformers. arXiv preprint arXiv:2107.00910.","DOI":"10.1145\/3534678.3539260"},{"key":"e_1_3_2_1_26_1","volume-title":"Kingma and Jimmy Ba","author":"Diederik","year":"2017","unstructured":"Diederik P. Kingma and Jimmy Ba . 2017 . Adam : a method for stochastic optimization. arxiv:1412.6980. Diederik P. Kingma and Jimmy Ba. 2017. Adam: a method for stochastic optimization. arxiv:1412.6980."},{"key":"e_1_3_2_1_27_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.703"},{"key":"e_1_3_2_1_28_1","unstructured":"Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy Mike Lewis Luke Zettlemoyer and Veselin Stoyanov. 2019. Roberta: a robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692. \t\t\t\t  Yinhan Liu Myle Ott Naman Goyal Jingfei Du Mandar Joshi Danqi Chen Omer Levy Mike Lewis Luke Zettlemoyer and Veselin Stoyanov. 2019. Roberta: a robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692."},{"key":"e_1_3_2_1_29_1","volume-title":"CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1).","author":"Lu Shuai","year":"2021","unstructured":"Shuai Lu , Daya Guo , Shuo Ren , Junjie Huang , Alexey Svyatkovskiy , Ambrosio Blanco , Colin Clement , Dawn Drain , Daxin Jiang , and Duyu Tang . 2021 . CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1). Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin Clement, Dawn Drain, Daxin Jiang, and Duyu Tang. 2021. CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 1)."},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE43902.2021.00041"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSC.2016.2560165"},{"key":"e_1_3_2_1_32_1","volume-title":"Comparing the Attention of Humans with Neural Models of Code. In 2021 36th IEEE\/ACM International Conference on Automated Software Engineering (ASE). 867\u2013879","author":"Paltenghi Matteo","year":"2021","unstructured":"Matteo Paltenghi and Michael Pradel . 2021 . Thinking Like a Developer? Comparing the Attention of Humans with Neural Models of Code. In 2021 36th IEEE\/ACM International Conference on Automated Software Engineering (ASE). 867\u2013879 . Matteo Paltenghi and Michael Pradel. 2021. Thinking Like a Developer? Comparing the Attention of Humans with Neural Models of Code. In 2021 36th IEEE\/ACM International Conference on Automated Software Engineering (ASE). 867\u2013879."},{"key":"e_1_3_2_1_33_1","unstructured":"David Pisinger. 1995. Algorithms for knapsack problems. \t\t\t\t  David Pisinger. 1995. Algorithms for knapsack problems."},{"key":"e_1_3_2_1_34_1","volume-title":"Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC\/FSE). Association for Computing Machinery, 441\u2013452","author":"Islam Rabin Md Rafiqul","year":"2021","unstructured":"Md Rafiqul Islam Rabin , Vincent J. Hellendoorn , and Mohammad Amin Alipour . 2021 . Understanding neural code intelligence through program simplification . In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC\/FSE). Association for Computing Machinery, 441\u2013452 . Md Rafiqul Islam Rabin, Vincent J. Hellendoorn, and Mohammad Amin Alipour. 2021. Understanding neural code intelligence through program simplification. In Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering (ESEC\/FSE). Association for Computing Machinery, 441\u2013452."},{"key":"e_1_3_2_1_35_1","volume-title":"Language models are unsupervised multitask learners. OpenAI blog, 1, 8","author":"Radford Alec","year":"2019","unstructured":"Alec Radford , Jeffrey Wu , Rewon Child , David Luan , Dario Amodei , and Ilya Sutskever . 2019. Language models are unsupervised multitask learners. OpenAI blog, 1, 8 ( 2019 ), 9. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1, 8 (2019), 9."},{"key":"e_1_3_2_1_36_1","first-page":"1","article-title":"Exploring the limits of transfer learning with a unified text-to-text transformer","volume":"21","author":"Raffel Colin","year":"2020","unstructured":"Colin Raffel , Noam Shazeer , Adam Roberts , Katherine Lee , Sharan Narang , Michael Matena , Yanqi Zhou , Wei Li , and Peter J Liu . 2020 . Exploring the limits of transfer learning with a unified text-to-text transformer . Journal of Machine Learning Research , 21 (2020), 1 \u2013 67 . Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21 (2020), 1\u201367.","journal-title":"Journal of Machine Learning Research"},{"key":"e_1_3_2_1_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/2568225.2568247"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1162\/tacl_a_00349"},{"key":"e_1_3_2_1_39_1","unstructured":"Victor Sanh Lysandre Debut Julien Chaumond and Thomas Wolf. 2019. DistilBERT a distilled version of BERT: smaller faster cheaper and lighter. arXiv preprint arXiv:1910.01108. \t\t\t\t  Victor Sanh Lysandre Debut Julien Chaumond and Thomas Wolf. 2019. DistilBERT a distilled version of BERT: smaller faster cheaper and lighter. arXiv preprint arXiv:1910.01108."},{"key":"e_1_3_2_1_40_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i05.6409"},{"key":"e_1_3_2_1_41_1","doi-asserted-by":"publisher","DOI":"10.1145\/2581377"},{"key":"e_1_3_2_1_42_1","doi-asserted-by":"publisher","DOI":"10.1145\/3468264.3468545"},{"key":"e_1_3_2_1_43_1","volume-title":"\u0141 ukasz Kaiser, and Illia Polosukhin","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani , Noam Shazeer , Niki Parmar , Jakob Uszkoreit , Llion Jones , Aidan N Gomez , \u0141 ukasz Kaiser, and Illia Polosukhin . 2017 . Attention is all you need. In Advances in neural information processing systems (NeurlPS 2017). 5998\u20136008. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, \u0141 ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems (NeurlPS 2017). 5998\u20136008."},{"key":"e_1_3_2_1_44_1","volume-title":"In Proceedings of the 44th International Conference on Software Engineering (ICSE).","author":"Wan Yao","year":"2022","unstructured":"Yao Wan , Wei Zhao , Hongyu Zhang , Yulei Sui , Guandong Xu , and Hai Jin . 2022 . What do they capture? \u2013 a structural analysis of pre-trained language models for source code . In In Proceedings of the 44th International Conference on Software Engineering (ICSE). Yao Wan, Wei Zhao, Hongyu Zhang, Yulei Sui, Guandong Xu, and Hai Jin. 2022. What do they capture? \u2013 a structural analysis of pre-trained language models for source code. In In Proceedings of the 44th International Conference on Software Engineering (ICSE)."},{"key":"e_1_3_2_1_45_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.emnlp-main.685"},{"key":"e_1_3_2_1_46_1","doi-asserted-by":"publisher","DOI":"10.1145\/2970276.2970326"},{"key":"e_1_3_2_1_47_1","volume-title":"Xlnet: generalized autoregressive pretraining for language understanding. Advances in neural information processing systems, 32","author":"Yang Zhilin","year":"2019","unstructured":"Zhilin Yang , Zihang Dai , Yiming Yang , Jaime Carbonell , Russ R Salakhutdinov , and Quoc V Le. 2019. Xlnet: generalized autoregressive pretraining for language understanding. Advances in neural information processing systems, 32 ( 2019 ). Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Russ R Salakhutdinov, and Quoc V Le. 2019. Xlnet: generalized autoregressive pretraining for language understanding. Advances in neural information processing systems, 32 (2019)."},{"key":"e_1_3_2_1_48_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2021.naacl-main.463"},{"key":"e_1_3_2_1_49_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.988498"},{"key":"e_1_3_2_1_50_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00086"}],"event":{"name":"ESEC\/FSE '22: 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering","location":"Singapore Singapore","acronym":"ESEC\/FSE '22","sponsor":["SIGSOFT ACM Special Interest Group on Software Engineering","NUS NUS"]},"container-title":["Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3540250.3549094","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3540250.3549094","content-type":"application\/pdf","content-version":"vor","intended-application":"syndication"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3540250.3549094","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T17:51:02Z","timestamp":1750182662000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3540250.3549094"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,11,7]]},"references-count":50,"alternative-id":["10.1145\/3540250.3549094","10.1145\/3540250"],"URL":"https:\/\/doi.org\/10.1145\/3540250.3549094","relation":{},"subject":[],"published":{"date-parts":[[2022,11,7]]},"assertion":[{"value":"2022-11-09","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}