{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,2,20]],"date-time":"2026-02-20T07:35:38Z","timestamp":1771572938139,"version":"3.50.1"},"publisher-location":"New York, NY, USA","reference-count":39,"publisher":"ACM","license":[{"start":{"date-parts":[[2021,8,18]],"date-time":"2021-08-18T00:00:00Z","timestamp":1629244800000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by-nd\/4.0\/"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":[],"published-print":{"date-parts":[[2021,8,20]]},"DOI":"10.1145\/3468264.3468611","type":"proceedings-article","created":{"date-parts":[[2021,8,19]],"date-time":"2021-08-19T01:44:18Z","timestamp":1629337458000},"page":"703-715","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":50,"title":["Empirical study of transformers for source code"],"prefix":"10.1145","author":[{"ORCID":"https:\/\/orcid.org\/0000-0001-8188-3391","authenticated-orcid":false,"given":"Nadezhda","family":"Chirkova","sequence":"first","affiliation":[{"name":"HSE University, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-8439-390X","authenticated-orcid":false,"given":"Sergey","family":"Troshin","sequence":"additional","affiliation":[{"name":"HSE University, Russia"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2021,8,18]]},"reference":[{"key":"e_1_3_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.acl-main.449"},{"key":"e_1_3_2_1_2_1","doi-asserted-by":"publisher","DOI":"10.1145\/3183377.3183383"},{"key":"e_1_3_2_1_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/3359591.3359735"},{"key":"e_1_3_2_1_4_1","volume-title":"6th International Conference on Learning Representations, ICLR","author":"Allamanis Miltiadis","year":"2018","unstructured":"Miltiadis Allamanis , Marc Brockschmidt , and Mahmoud Khademi . 2018. Learning to Represent Programs with Graphs . In 6th International Conference on Learning Representations, ICLR 2018 , Vancouver, BC , Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview .net. https:\/\/openreview.net\/forum?id=BJOFETxR- Miltiadis Allamanis, Marc Brockschmidt, and Mahmoud Khademi. 2018. Learning to Represent Programs with Graphs. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings. OpenReview.net. https:\/\/openreview.net\/forum?id=BJOFETxR-"},{"key":"e_1_3_2_1_5_1","volume-title":"International Conference on Machine Learning (ICML).","author":"Allamanis Miltiadis","year":"2016","unstructured":"Miltiadis Allamanis , Hao Peng , and Charles Sutton . 2016 . A Convolutional Attention Network for Extreme Summarization of Source Code . In International Conference on Machine Learning (ICML). Miltiadis Allamanis, Hao Peng, and Charles Sutton. 2016. A Convolutional Attention Network for Extreme Summarization of Source Code. In International Conference on Machine Learning (ICML)."},{"key":"e_1_3_2_1_6_1","volume-title":"7th International Conference on Learning Representations, ICLR 2019","author":"Alon Uri","year":"2019","unstructured":"Uri Alon , Shaked Brody , Omer Levy , and Eran Yahav . 2019 . code2seq: Generating Sequences from Structured Representations of Code . In 7th International Conference on Learning Representations, ICLR 2019 , New Orleans, LA, USA , May 6-9, 2019. OpenReview.net. https:\/\/openreview.net\/forum?id=H1gKYo09tX Uri Alon, Shaked Brody, Omer Levy, and Eran Yahav. 2019. code2seq: Generating Sequences from Structured Representations of Code. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net. https:\/\/openreview.net\/forum?id=H1gKYo09tX"},{"key":"e_1_3_2_1_7_1","volume-title":"International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=H1gKYo09tX","author":"Alon Uri","year":"2019","unstructured":"Uri Alon , Shaked Brody , Omer Levy , and Eran Yahav . 2019 . code2seq: Generating Sequences from Structured Representations of Code . In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=H1gKYo09tX Uri Alon, Shaked Brody, Omer Levy, and Eran Yahav. 2019. code2seq: Generating Sequences from Structured Representations of Code. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=H1gKYo09tX"},{"key":"e_1_3_2_1_8_1","volume-title":"Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning. In International Conference on Learning Representations, ICLR","author":"Ashukha Arsenii","year":"2020","unstructured":"Arsenii Ashukha , Alexander Lyzhov , Dmitry Molchanov , and Dmitry Vetrov . 2020 . Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning. In International Conference on Learning Representations, ICLR 2020. https:\/\/openreview.net\/forum?id=BJxI5gHKDr Arsenii Ashukha, Alexander Lyzhov, Dmitry Molchanov, and Dmitry Vetrov. 2020. Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning. In International Conference on Learning Representations, ICLR 2020. https:\/\/openreview.net\/forum?id=BJxI5gHKDr"},{"key":"e_1_3_2_1_9_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N19-1423"},{"key":"e_1_3_2_1_10_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/2020.findings-emnlp.139"},{"key":"e_1_3_2_1_11_1","volume-title":"Structured Neural Summarization. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=H1ersoRqtm","author":"Fernandes Patrick","year":"2019","unstructured":"Patrick Fernandes , Miltiadis Allamanis , and Marc Brockschmidt . 2019 . Structured Neural Summarization. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=H1ersoRqtm Patrick Fernandes, Miltiadis Allamanis, and Marc Brockschmidt. 2019. Structured Neural Summarization. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=H1ersoRqtm"},{"key":"e_1_3_2_1_12_1","volume-title":"GraphCodeBERT: Pre-training Code Representations with Data Flow. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=jLoC4ez43PZ","author":"Guo Daya","year":"2021","unstructured":"Daya Guo , Shuo Ren , Shuai Lu , Zhangyin Feng , Duyu Tang , Shujie LIU , Long Zhou , Nan Duan , Alexey Svyatkovskiy , Shengyu Fu , Michele Tufano , Shao Kun Deng , Colin Clement , Dawn Drain , Neel Sundaresan , Jian Yin , Daxin Jiang , and Ming Zhou . 2021 . GraphCodeBERT: Pre-training Code Representations with Data Flow. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=jLoC4ez43PZ Daya Guo, Shuo Ren, Shuai Lu, Zhangyin Feng, Duyu Tang, Shujie LIU, Long Zhou, Nan Duan, Alexey Svyatkovskiy, Shengyu Fu, Michele Tufano, Shao Kun Deng, Colin Clement, Dawn Drain, Neel Sundaresan, Jian Yin, Daxin Jiang, and Ming Zhou. 2021. GraphCodeBERT: Pre-training Code Representations with Data Flow. In International Conference on Learning Representations. https:\/\/openreview.net\/forum?id=jLoC4ez43PZ"},{"key":"e_1_3_2_1_13_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v31i1.10742"},{"key":"e_1_3_2_1_14_1","volume-title":"Global Relational Models of Source Code. In International Conference on Learning Representations, ICLR","author":"Hellendoorn Vincent J.","year":"2020","unstructured":"Vincent J. Hellendoorn , Charles Sutton , Rishabh Singh , Petros Maniatis , and David Bieber . 2020 . Global Relational Models of Source Code. In International Conference on Learning Representations, ICLR 2020. https:\/\/openreview.net\/forum?id=B1lnbRNtwr Vincent J. Hellendoorn, Charles Sutton, Rishabh Singh, Petros Maniatis, and David Bieber. 2020. Global Relational Models of Source Code. In International Conference on Learning Representations, ICLR 2020. https:\/\/openreview.net\/forum?id=B1lnbRNtwr"},{"key":"e_1_3_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1162\/neco.1997.9.8.1735"},{"key":"e_1_3_2_1_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/3196321.3196334"},{"key":"e_1_3_2_1_17_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/P16-1195"},{"key":"e_1_3_2_1_18_1","volume-title":"Proceedings of the 37th International Conference on Machine Learning, ICML 2020","author":"Kanade Aditya","year":"2020","unstructured":"Aditya Kanade , Petros Maniatis , Gogul Balakrishnan , and Kensen Shi . 2020 . Learning and evaluating contextual embedding of source code . In Proceedings of the 37th International Conference on Machine Learning, ICML 2020 , 12-18 July 2020 (Proceedings of Machine Learning Research). PMLR. Aditya Kanade, Petros Maniatis, Gogul Balakrishnan, and Kensen Shi. 2020. Learning and evaluating contextual embedding of source code. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 12-18 July 2020 (Proceedings of Machine Learning Research). PMLR."},{"key":"e_1_3_2_1_19_1","doi-asserted-by":"publisher","DOI":"10.1145\/3377811.3380342"},{"key":"e_1_3_2_1_20_1","unstructured":"Seohyun Kim Jinman Zhao Yuchi Tian and Satish Chandra. 2020. Code Prediction by Feeding Trees to Transformers. arxiv:2003.13848.  Seohyun Kim Jinman Zhao Yuchi Tian and Satish Chandra. 2020. Code Prediction by Feeding Trees to Transformers. arxiv:2003.13848."},{"key":"e_1_3_2_1_21_1","unstructured":"Marie-Anne Lachaux Baptiste Roziere Lowik Chanussot and Guillaume Lample. 2020. Unsupervised Translation of Programming Languages. In arXiv preprint arXiv:2006.03511. arxiv:2006.03511.  Marie-Anne Lachaux Baptiste Roziere Lowik Chanussot and Guillaume Lample. 2020. Unsupervised Translation of Programming Languages. In arXiv preprint arXiv:2006.03511. arxiv:2006.03511."},{"key":"e_1_3_2_1_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2019.00087"},{"key":"e_1_3_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.5555\/3304222.3304348"},{"key":"e_1_3_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2018\/578"},{"key":"e_1_3_2_1_25_1","volume-title":"SGDR: Stochastic Gradient Descent with Warm Restarts. In ICLR.","author":"Loshchilov I.","year":"2017","unstructured":"I. Loshchilov and F. Hutter . 2017 . SGDR: Stochastic Gradient Descent with Warm Restarts. In ICLR. I. Loshchilov and F. Hutter. 2017. SGDR: Stochastic Gradient Descent with Warm Restarts. In ICLR."},{"key":"e_1_3_2_1_26_1","volume-title":"Proceedings of the 31st International Conference on Machine Learning, Eric P. Xing and Tony Jebara (Eds.) (Proceedings of Machine Learning Research","volume":"657","author":"Maddison Chris","year":"2014","unstructured":"Chris Maddison and Daniel Tarlow . 2014 . Structured Generative Models of Natural Source Code . In Proceedings of the 31st International Conference on Machine Learning, Eric P. Xing and Tony Jebara (Eds.) (Proceedings of Machine Learning Research , Vol. 32). PMLR, Bejing, China. 649\u2013 657 . http:\/\/proceedings.mlr.press\/v32\/maddison14.html Chris Maddison and Daniel Tarlow. 2014. Structured Generative Models of Natural Source Code. In Proceedings of the 31st International Conference on Machine Learning, Eric P. Xing and Tony Jebara (Eds.) (Proceedings of Machine Learning Research, Vol. 32). PMLR, Bejing, China. 649\u2013657. http:\/\/proceedings.mlr.press\/v32\/maddison14.html"},{"key":"e_1_3_2_1_27_1","unstructured":"Martin Monperrus. 2020. The Living Review on Automated Program Repair. Dec. https:\/\/hal.archives-ouvertes.fr\/hal-01956501 working paper or preprint.  Martin Monperrus. 2020. The Living Review on Automated Program Repair. Dec. https:\/\/hal.archives-ouvertes.fr\/hal-01956501 working paper or preprint."},{"key":"e_1_3_2_1_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/2983990.2984041"},{"key":"e_1_3_2_1_29_1","doi-asserted-by":"publisher","DOI":"10.1145\/3022671.2984041"},{"key":"e_1_3_2_1_30_1","doi-asserted-by":"publisher","DOI":"10.1145\/2837614.2837671"},{"key":"e_1_3_2_1_31_1","doi-asserted-by":"publisher","DOI":"10.18653\/v1\/N18-2074"},{"key":"e_1_3_2_1_32_1","volume-title":"Advances in Neural Information Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. d' Alch\u00e9-Buc","author":"Shiv Vighnesh","year":"2081","unstructured":"Vighnesh Shiv and Chris Quirk . 2019. Novel positional encodings to enable tree-based transformers . In Advances in Neural Information Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. d' Alch\u00e9-Buc , E. Fox, and R. Garnett (Eds.). Curran Associates, Inc. , 1 2081 \u201312091. http:\/\/papers.nips.cc\/paper\/9376-novel-positional-encodings-to-enable-tree-based-transformers.pdf Vighnesh Shiv and Chris Quirk. 2019. Novel positional encodings to enable tree-based transformers. In Advances in Neural Information Processing Systems 32, H. Wallach, H. Larochelle, A. Beygelzimer, F. d' Alch\u00e9-Buc, E. Fox, and R. Garnett (Eds.). Curran Associates, Inc., 12081\u201312091. http:\/\/papers.nips.cc\/paper\/9376-novel-positional-encodings-to-enable-tree-based-transformers.pdf"},{"key":"e_1_3_2_1_33_1","doi-asserted-by":"publisher","DOI":"10.1609\/aaai.v34i05.6430"},{"key":"e_1_3_2_1_34_1","doi-asserted-by":"publisher","DOI":"10.3115\/v1\/P15-1150"},{"key":"e_1_3_2_1_35_1","volume-title":"Efficient Transformers: A Survey. arxiv:2009.06732.","author":"Tay Yi","year":"2020","unstructured":"Yi Tay , Mostafa Dehghani , Dara Bahri , and Donald Metzler . 2020 . Efficient Transformers: A Survey. arxiv:2009.06732. Yi Tay, Mostafa Dehghani, Dara Bahri, and Donald Metzler. 2020. Efficient Transformers: A Survey. arxiv:2009.06732."},{"key":"e_1_3_2_1_36_1","volume-title":"7th International Conference on Learning Representations, ICLR 2019","author":"Vasic Marko","year":"2019","unstructured":"Marko Vasic , Aditya Kanade , Petros Maniatis , David Bieber , and Rishabh Singh . 2019 . Neural Program Repair by Jointly Learning to Localize and Repair . In 7th International Conference on Learning Representations, ICLR 2019 , New Orleans, LA, USA , May 6-9, 2019. OpenReview.net. https:\/\/openreview.net\/forum?id=ByloJ20qtm Marko Vasic, Aditya Kanade, Petros Maniatis, David Bieber, and Rishabh Singh. 2019. Neural Program Repair by Jointly Learning to Localize and Repair. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net. https:\/\/openreview.net\/forum?id=ByloJ20qtm"},{"key":"e_1_3_2_1_37_1","first-page":"I","article-title":"Attention is All you Need","volume":"30","author":"Vaswani Ashish","year":"2017","unstructured":"Ashish Vaswani , Noam Shazeer , Niki Parmar , Jakob Uszkoreit , Llion Jones , Aidan N Gomez , \u0141 ukasz Kaiser, and Illia Polosukhin . 2017 . Attention is All you Need . In Advances in Neural Information Processing Systems 30 , I . Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.). Curran Associates, Inc., 5998\u20136008. http:\/\/papers.nips.cc\/paper\/7181-attention-is-all-you-need.pdf Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, \u0141 ukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Advances in Neural Information Processing Systems 30, I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (Eds.). Curran Associates, Inc., 5998\u20136008. http:\/\/papers.nips.cc\/paper\/7181-attention-is-all-you-need.pdf","journal-title":"Advances in Neural Information Processing Systems"},{"key":"e_1_3_2_1_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/3238147.3238206"},{"key":"e_1_3_2_1_39_1","doi-asserted-by":"publisher","DOI":"10.24963\/ijcai.2019\/552"}],"event":{"name":"ESEC\/FSE '21: 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering","location":"Athens Greece","acronym":"ESEC\/FSE '21","sponsor":["SIGSOFT ACM Special Interest Group on Software Engineering"]},"container-title":["Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering"],"original-title":[],"link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3468264.3468611","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/3468264.3468611","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,17]],"date-time":"2025-06-17T20:17:22Z","timestamp":1750191442000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/3468264.3468611"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,8,18]]},"references-count":39,"alternative-id":["10.1145\/3468264.3468611","10.1145\/3468264"],"URL":"https:\/\/doi.org\/10.1145\/3468264.3468611","relation":{},"subject":[],"published":{"date-parts":[[2021,8,18]]},"assertion":[{"value":"2021-08-18","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}