{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,28]],"date-time":"2026-01-28T06:16:21Z","timestamp":1769580981497,"version":"3.49.0"},"reference-count":54,"publisher":"World Scientific Pub Co Pte Lt","issue":"05","funder":[{"name":"the National Key R&D","award":["2018YFB1003902"],"award-info":[{"award-number":["2018YFB1003902"]}]},{"DOI":"10.13039\/501100004608","name":"Natural Science Foundation of Jiangsu Province","doi-asserted-by":"crossref","award":["BK20170809"],"award-info":[{"award-number":["BK20170809"]}],"id":[{"id":"10.13039\/501100004608","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61972197"],"award-info":[{"award-number":["61972197"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]},{"DOI":"10.13039\/501100001809","name":"National Natural Science Foundation of China","doi-asserted-by":"crossref","award":["61802179"],"award-info":[{"award-number":["61802179"]}],"id":[{"id":"10.13039\/501100001809","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":["Int. J. Soft. Eng. Knowl. Eng."],"published-print":{"date-parts":[[2020,5]]},"abstract":"<jats:p>In recent years, deep learning models have shown great potential in source code modeling and analysis. Generally, deep learning-based approaches are problem-specific and data-hungry. A challenging issue of these approaches is that they require training from scratch for a different related problem. In this work, we propose a transfer learning-based approach that significantly improves the performance of deep learning-based source code models. In contrast to traditional learning paradigms, transfer learning can transfer the knowledge learned in solving one problem into another related problem. First, we present two recurrent neural network-based models RNN and GRU for the purpose of transfer learning in the domain of source code modeling. Next, via transfer learning, these pre-trained (RNN and GRU) models are used as feature extractors. Then, these extracted features are combined into attention learner for different downstream tasks. The attention learner leverages from the learned knowledge of pre-trained models and fine-tunes them for a specific downstream task. We evaluate the performance of the proposed approach with extensive experiments with the source code suggestion task. The results indicate that the proposed approach outperforms the state-of-the-art models in terms of accuracy, precision, recall and F-measure without training the models from scratch.<\/jats:p>","DOI":"10.1142\/s0218194020500230","type":"journal-article","created":{"date-parts":[[2020,6,26]],"date-time":"2020-06-26T11:28:04Z","timestamp":1593170884000},"page":"649-668","source":"Crossref","is-referenced-by-count":19,"title":["Deep Transfer Learning for Source Code Modeling"],"prefix":"10.1142","volume":"30","author":[{"given":"Yasir","family":"Hussain","sequence":"first","affiliation":[{"name":"College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics (NUAA), Nanjing, Jiangsu 211106, P.\u00a0R.\u00a0China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Zhiqiu","family":"Huang","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics (NUAA), Nanjing, Jiangsu 211106, P.\u00a0R.\u00a0China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Yu","family":"Zhou","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics (NUAA), Nanjing, Jiangsu 211106, P.\u00a0R.\u00a0China"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Senzhang","family":"Wang","sequence":"additional","affiliation":[{"name":"College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics (NUAA), Nanjing, Jiangsu 211106, P.\u00a0R.\u00a0China"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"219","published-online":{"date-parts":[[2020,6,26]]},"reference":[{"key":"S0218194020500230BIB001","first-page":"2091","volume-title":"Int. Conf. Machine Learning","author":"Allamanis M.","year":"2016"},{"key":"S0218194020500230BIB002","first-page":"40","volume-title":"Proc. ACM on Programming Languages","volume":"3","author":"Alon U.","year":"2019"},{"key":"S0218194020500230BIB003","first-page":"311","volume-title":"2018 IEEE 25th Int. Conf. Software Analysis, Evolution and Reengineering","author":"Santos E. A.","year":"2018"},{"key":"S0218194020500230BIB005","doi-asserted-by":"crossref","first-page":"2073","DOI":"10.18653\/v1\/P16-1195","volume-title":"Proc. 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)","author":"Iyer S.","year":"2016"},{"issue":"12","key":"S0218194020500230BIB006","doi-asserted-by":"crossref","first-page":"1095","DOI":"10.1109\/TSE.2017.2664836","volume":"43","author":"Fowkes J.","year":"2017","journal-title":"IEEE Trans. Softw. Eng."},{"issue":"6","key":"S0218194020500230BIB007","doi-asserted-by":"crossref","first-page":"419","DOI":"10.1145\/2666356.2594321","volume":"49","author":"Raychev V.","year":"2014","journal-title":"Acm Sigplan Notices"},{"key":"S0218194020500230BIB008","volume-title":"32nd AAAI Conf. Artificial Intelligence","author":"Sethi A.","year":"2018"},{"key":"S0218194020500230BIB009","first-page":"334","volume-title":"Proc. 12th Working Conf. Mining Software Repositories","author":"White M.","year":"2015"},{"issue":"1","key":"S0218194020500230BIB011","doi-asserted-by":"crossref","first-page":"3","DOI":"10.3390\/computers8010003","volume":"8","author":"Salem M.","year":"2019","journal-title":"Computers"},{"key":"S0218194020500230BIB012","first-page":"837","volume-title":"2012 34th Int. Conf. Software Engineering","author":"Hindle A.","year":"2012"},{"key":"S0218194020500230BIB013","doi-asserted-by":"publisher","DOI":"10.1145\/2635868.2635875"},{"key":"S0218194020500230BIB014","doi-asserted-by":"crossref","first-page":"763","DOI":"10.1145\/3106237.3106290","volume-title":"Proc. 2017 11th Joint Meeting on Foundations of Software Engineering","author":"Hellendoorn V. J.","year":"2017"},{"key":"S0218194020500230BIB015","first-page":"2933","volume-title":"Int. Conf. Machine Learning","author":"Bielik P.","year":"2016"},{"key":"S0218194020500230BIB016","first-page":"472","volume-title":"Proc. 22nd ACM SIGSOFT Int. Symp. Foundations of Software Engineering","author":"Allamanis M.","year":"2014"},{"key":"S0218194020500230BIB017","first-page":"649","volume-title":"Int. Conf. Machine Learning","author":"Maddison C.","year":"2014"},{"issue":"1","key":"S0218194020500230BIB018","doi-asserted-by":"crossref","first-page":"761","DOI":"10.1145\/2914770.2837671","volume":"51","author":"Raychev V.","year":"2016","journal-title":"ACM Sigplan Notices"},{"key":"S0218194020500230BIB019","first-page":"10","volume-title":"Proc. ACM SIGSOFT 20th Int. Symp. Foundations of Software Engineering","author":"Chan W.-K.","year":"2012"},{"key":"S0218194020500230BIB020","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2013.6624045"},{"key":"S0218194020500230BIB021","first-page":"664","volume-title":"Proc. 36th Int. Conf. Software Engineering","author":"Keivanloo I.","year":"2014"},{"key":"S0218194020500230BIB022","first-page":"51","volume-title":"2016 IEEE 16th Int. Working Conf. Source Code Analysis and Manipulation","author":"D\u2019Souza A. R.","year":"2016"},{"key":"S0218194020500230BIB023","first-page":"290","volume-title":"Proc. 28th IEEE\/ACM Int. Conf. Automated Software Engineering","author":"Thung F.","year":"2013"},{"key":"S0218194020500230BIB024","first-page":"416","volume-title":"Proc. 38th Int. Conf. Software Engineering","author":"Pham H. V.","year":"2016"},{"key":"S0218194020500230BIB026","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1145\/2786805.2786849","volume-title":"Proc. 2015 10th Joint Meeting on Foundations of Software Engineering","author":"Allamanis M.","year":"2015"},{"key":"S0218194020500230BIB027","first-page":"705","volume-title":"Proc. 37th Int. Conf. Software Engineering-Volume 2","author":"Franks C.","year":"2015"},{"key":"S0218194020500230BIB028","first-page":"69","volume-title":"2012 34th Int. Conf. Software Engineering","author":"Nguyen A. T.","year":"2012"},{"key":"S0218194020500230BIB029","doi-asserted-by":"crossref","first-page":"834","DOI":"10.1145\/2970276.2970290","volume-title":"2016 31st IEEE\/ACM Int. Conf. Automated Software Engineering","author":"Chen C.","year":"2016"},{"key":"S0218194020500230BIB031","first-page":"2123","volume-title":"Int. Conf. Machine Learning","author":"Allamanis M.","year":"2015"},{"key":"S0218194020500230BIB032","doi-asserted-by":"crossref","first-page":"639","DOI":"10.1109\/ICSE.2015.212","volume-title":"2015 IEEE\/ACM 37th IEEE Int. Conf. Software Engineering","volume":"2","author":"Guerrouj L.","year":"2015"},{"key":"S0218194020500230BIB033","doi-asserted-by":"publisher","DOI":"10.1142\/S0218194017500322"},{"key":"S0218194020500230BIB034","doi-asserted-by":"crossref","first-page":"54","DOI":"10.1145\/1137983.1137997","volume-title":"Proc. 2006 Int. Workshop on Mining Software Repositories","author":"Xie T.","year":"2006"},{"issue":"2","key":"S0218194020500230BIB035","first-page":"3","volume":"1","author":"Kumar B.","year":"2015","journal-title":"Complexity"},{"key":"S0218194020500230BIB036","doi-asserted-by":"crossref","first-page":"1469","DOI":"10.1016\/j.neucom.2014.08.045","volume":"149","author":"Zhang X.","year":"2015","journal-title":"Neurocomput."},{"key":"S0218194020500230BIB037","first-page":"631","volume-title":"Proc. 2016 24th ACM SIGSOFT Int. Symp. Foundations of Software Engineering","author":"Gu X.","year":"2016"},{"key":"S0218194020500230BIB038","doi-asserted-by":"crossref","first-page":"1246","DOI":"10.1145\/2623330.2623728","volume-title":"Proc. 20th ACM SIGKDD Int. Conf. Knowledge Discovery and Data Mining","author":"Wang S.","year":"2014"},{"key":"S0218194020500230BIB039","doi-asserted-by":"crossref","first-page":"328","DOI":"10.1016\/j.jss.2019.07.087","volume":"156","author":"Zhou Y.","year":"2019","journal-title":"J. Syst. Soft."},{"key":"S0218194020500230BIB040","first-page":"325","volume-title":"Proc. IEEE Conf. Computer Vision and Pattern Recognition","author":"Hu J.","year":"2015"},{"key":"S0218194020500230BIB041","first-page":"1375","volume-title":"Proc. IEEE Int. Computer Version Pattern Recognit.","author":"Duan L.","year":"2009"},{"key":"S0218194020500230BIB042","first-page":"677","volume-title":"Proc. 23 National Conf. Artifical Intelligence","volume":"8","author":"Pan S. J.","year":"2008"},{"key":"S0218194020500230BIB043","doi-asserted-by":"crossref","first-page":"1","DOI":"10.1016\/j.patrec.2019.03.022","volume":"125","author":"Khan S.","year":"2019","journal-title":"Pattern Recognit. Lett."},{"key":"S0218194020500230BIB044","doi-asserted-by":"publisher","DOI":"10.3390\/rs9090907"},{"key":"S0218194020500230BIB045","doi-asserted-by":"crossref","first-page":"38","DOI":"10.1016\/j.dss.2017.10.001","volume":"104","author":"Kraus M.","year":"2017","journal-title":"Decis. Support Syst."},{"key":"S0218194020500230BIB046","doi-asserted-by":"publisher","DOI":"10.1109\/TMI.2016.2528162"},{"key":"S0218194020500230BIB047","first-page":"1","author":"Zhang W.","year":"2016","journal-title":"IEEE Trans. Big Data"},{"issue":"5","key":"S0218194020500230BIB048","first-page":"1963","volume":"10","author":"Yuan Y.","year":"2017","journal-title":"IEEE J. STARS"},{"key":"S0218194020500230BIB049","doi-asserted-by":"publisher","DOI":"10.1109\/MCI.2018.2840738"},{"key":"S0218194020500230BIB050","first-page":"577","volume-title":"29 Annual Conf. Adv. Neural Information Processing Systems","author":"Chorowski J. K.","year":"2015"},{"key":"S0218194020500230BIB052","doi-asserted-by":"crossref","first-page":"4945","DOI":"10.1109\/ICASSP.2016.7472618","volume-title":"2016 IEEE Int. Conf. Acoustics, Speech and Signal Processing","author":"Bahdanau D.","year":"2016"},{"key":"S0218194020500230BIB053","first-page":"5998","volume-title":"31st Conf. Adv. Neural Inf. Process. Syst.","author":"Vaswani A.","year":"2017"},{"key":"S0218194020500230BIB054","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2015.7298594"},{"key":"S0218194020500230BIB055","doi-asserted-by":"publisher","DOI":"10.1109\/CVPR.2016.90"},{"issue":"8","key":"S0218194020500230BIB058","first-page":"9","volume":"1","author":"Radford A.","year":"2019","journal-title":"OpenAI Blog"},{"key":"S0218194020500230BIB059","first-page":"323","volume-title":"2018 IEEE 25th Int. Conf. Software Analysis, Evolution and Reengineering","author":"Nguyen A. T.","year":"2018"},{"key":"S0218194020500230BIB061","first-page":"13","volume-title":"Workshop on Recommender Systems and Big Data Analytics","author":"Matuszyk P.","year":"2016"},{"key":"S0218194020500230BIB062","first-page":"3371","volume":"11","author":"Vincent P.","year":"2010","journal-title":"J. Machine Learning Res."}],"container-title":["International Journal of Software Engineering and Knowledge Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/www.worldscientific.com\/doi\/pdf\/10.1142\/S0218194020500230","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,3,21]],"date-time":"2021-03-21T22:28:51Z","timestamp":1616365731000},"score":1,"resource":{"primary":{"URL":"https:\/\/www.worldscientific.com\/doi\/abs\/10.1142\/S0218194020500230"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2020,5]]},"references-count":54,"journal-issue":{"issue":"05","published-print":{"date-parts":[[2020,5]]}},"alternative-id":["10.1142\/S0218194020500230"],"URL":"https:\/\/doi.org\/10.1142\/s0218194020500230","relation":{},"ISSN":["0218-1940","1793-6403"],"issn-type":[{"value":"0218-1940","type":"print"},{"value":"1793-6403","type":"electronic"}],"subject":[],"published":{"date-parts":[[2020,5]]}}}