{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,6]],"date-time":"2026-03-06T11:52:46Z","timestamp":1772797966905,"version":"3.50.1"},"reference-count":75,"publisher":"Springer Science and Business Media LLC","issue":"10","license":[{"start":{"date-parts":[[2022,9,23]],"date-time":"2022-09-23T00:00:00Z","timestamp":1663891200000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2022,9,23]],"date-time":"2022-09-23T00:00:00Z","timestamp":1663891200000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"name":"Japan Society for the Promotion of Science (JSPS) KAKENHI","award":["19K12252"],"award-info":[{"award-number":["19K12252"]}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Appl Intell"],"published-print":{"date-parts":[[2023,5]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>In software, an algorithm is a well-organized sequence of actions that provides the optimal way to complete a task. Algorithmic thinking is also essential to break-down a problem and conceptualize solutions in some steps. The proper selection of an algorithm is pivotal to improve computational performance and software productivity as well as to programming learning. That is, determining a suitable algorithm from a given code is widely relevant in software engineering and programming education. However, both humans and machines find it difficult to identify algorithms from code without any meta-information. This study aims to propose a program code classification model that uses a convolutional neural network (CNN) to classify codes based on the algorithm. First, program codes are transformed into a sequence of structural features (SFs). Second, SFs are transformed into a one-hot binary matrix using several procedures. Third, different structures and hyperparameters of the CNN model are fine-tuned to identify the best model for the code classification task. To do so, 61,614 real-world program codes of different types of algorithms collected from an online judge system are used to train, validate, and evaluate the model. Finally, the experimental results show that the proposed model can identify algorithms and classify program codes with a high percentage of accuracy. The average precision, recall, and F-measure scores of the best CNN model are 95.65%, 95.85%, and 95.70%, respectively, indicating that it outperforms other baseline models.<\/jats:p>","DOI":"10.1007\/s10489-022-04078-y","type":"journal-article","created":{"date-parts":[[2022,9,23]],"date-time":"2022-09-23T15:06:21Z","timestamp":1663945581000},"page":"12210-12236","update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":28,"title":["Identifying algorithm in program code based on structural features using CNN classification model"],"prefix":"10.1007","volume":"53","author":[{"ORCID":"https:\/\/orcid.org\/0000-0002-0030-3859","authenticated-orcid":false,"given":"Yutaka","family":"Watanobe","sequence":"first","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0001-9368-7638","authenticated-orcid":false,"given":"Md. Mostafizer","family":"Rahman","sequence":"additional","affiliation":[]},{"given":"Md. Faizul Ibne","family":"Amin","sequence":"additional","affiliation":[]},{"ORCID":"https:\/\/orcid.org\/0000-0003-2031-8836","authenticated-orcid":false,"given":"Raihan","family":"Kabir","sequence":"additional","affiliation":[]}],"member":"297","published-online":{"date-parts":[[2022,9,23]]},"reference":[{"key":"4078_CR1","doi-asserted-by":"publisher","first-page":"139975","DOI":"10.1109\/ACCESS.2021.3119145","volume":"9","author":"MM Rahman","year":"2021","unstructured":"Rahman MM, Watanobe Y, Kiran RU, Thang TC, Paik I (2021) Impact of practical skills on academic performance: a data-driven analysis. IEEE Access 9:139975\u2013139993. https:\/\/doi.org\/10.1109\/ACCESS.2021.3119145https:\/\/doi.org\/10.1109\/ACCESS.2021.3119145","journal-title":"IEEE Access"},{"issue":"2","key":"4078_CR2","doi-asserted-by":"publisher","first-page":"77","DOI":"10.1109\/TE.2018.2864133","volume":"62","author":"RP Medeiros","year":"2019","unstructured":"Medeiros RP, Ramalho GL, Falc\u00e3o TP (2019) A systematic literature review on teaching and learning introductory programming in higher education. IEEE Trans Educ 62(2):77\u201390. https:\/\/doi.org\/10.1109\/TE.2018.2864133","journal-title":"IEEE Trans Educ"},{"key":"4078_CR3","doi-asserted-by":"publisher","first-page":"88121","DOI":"10.1109\/ACCESS.2021.3089560","volume":"9","author":"P Perera","year":"2021","unstructured":"Perera P, Tennakoon G, Ahangama S, Panditharathna R, Chathuranga B (2021) A systematic mapping of introductory programming languages for novice learners. IEEE Access 9:88121\u201388136. https:\/\/doi.org\/10.1109\/ACCESS.2021.3089560","journal-title":"IEEE Access"},{"key":"4078_CR4","doi-asserted-by":"publisher","first-page":"125961","DOI":"10.1109\/ACCESS.2020.3008321","volume":"8","author":"E Mehmood","year":"2020","unstructured":"Mehmood E, Abid A, Farooq MS, Nawaz NA (2020) Curriculum, teaching and learning, and assessments for introductory programming course. IEEE Access 8:125961\u2013125981. https:\/\/doi.org\/10.1109\/ACCESS.2020.3008321","journal-title":"IEEE Access"},{"issue":"06","key":"4078_CR5","doi-asserted-by":"publisher","first-page":"917","DOI":"10.1142\/S0218194022500346","volume":"32","author":"Y Watanobe","year":"2022","unstructured":"Watanobe Y, Rahman MM, Matsumoto T, Rage UK, Ravikumar P (2022) Online judge system: requirements, architecture, and experiences. Int J Softw Eng Knowl Eng 32(06):917\u2013946. https:\/\/doi.org\/10.1142\/S0218194022500346","journal-title":"Int J Softw Eng Knowl Eng"},{"key":"4078_CR6","doi-asserted-by":"publisher","unstructured":"Trisovic A, Lau MK, Pasquier T, Crosas M (2022) A large-scale study on research code quality and execution. Sci Data 9(1):. https:\/\/doi.org\/10.1038\/s41597-022-01143-6","DOI":"10.1038\/s41597-022-01143-6"},{"key":"4078_CR7","doi-asserted-by":"publisher","unstructured":"Teshima Y, Watanobe Y (2018) Bug detection based on lstm networks and solution codes. In: 2018 IEEE international conference on systems, man, and cybernetics (SMC), pp 3541\u20133546. https:\/\/doi.org\/10.1109\/SMC.2018.00599","DOI":"10.1109\/SMC.2018.00599"},{"issue":"8","key":"4078_CR8","doi-asserted-by":"publisher","first-page":"2973","DOI":"10.3390\/app10082973","volume":"10","author":"MM Rahman","year":"2020","unstructured":"Rahman MM, Watanobe Y, Nakamura K (2020) Source code assessment and classification based on estimated error probability using attentive lstm language model and its application in programming education. Appl Sci 10(8):2973. https:\/\/doi.org\/10.3390\/app10082973","journal-title":"Appl Sci"},{"key":"4078_CR9","doi-asserted-by":"publisher","unstructured":"Intisar CM, Watanobe Y (2018) Classification of online judge programmers based on rule extraction from self organizing feature map. In: 2018 9th international conference on awareness science and technology (iCAST), pp 313\u2013318. https:\/\/doi.org\/10.1109\/ICAwST.2018.8517222","DOI":"10.1109\/ICAwST.2018.8517222"},{"key":"4078_CR10","doi-asserted-by":"publisher","unstructured":"Intisar CM, Watanobe Y (2018) Cluster analysis to estimate the difficulty of programming problems. In: Proceedings of the 3rd international conference on applications in information technology. ICAIT\u20192018, pp 23\u201328. https:\/\/doi.org\/10.1145\/3274856.3274862","DOI":"10.1145\/3274856.3274862"},{"key":"4078_CR11","doi-asserted-by":"publisher","unstructured":"Rahman MM, Watanobe Y, Rage UK, Nakamura K (2021) A novel rule-based online judge recommender system to promote computer programming education. In: Fujita H, Selamat A, Lin JC-W, Ali M (eds) Advances and trends in artificial intelligence. From theory to practice, Springer, pp 15\u201327. https:\/\/doi.org\/10.1007\/978-3-030-79463-7_2","DOI":"10.1007\/978-3-030-79463-7_2"},{"issue":"1","key":"4078_CR12","doi-asserted-by":"publisher","first-page":"36","DOI":"10.4018\/IJDET.2020010103","volume":"18","author":"T Saito","year":"2020","unstructured":"Saito T, Watanobe Y (2020) Learning path recommendation system for programming education based on neural networks. Int J Dis Educ Technol (IJDET) 18(1):36\u201364. https:\/\/doi.org\/10.4018\/IJDET.2020010103","journal-title":"Int J Dis Educ Technol (IJDET)"},{"issue":"4","key":"4078_CR13","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/2492248.2492276","volume":"38","author":"F Taibi","year":"2013","unstructured":"Taibi F (2013) Reusability of open-source program code: a conceptual model and empirical investigation. SIGSOFT Softw. Eng. Notes 38(4):1\u20135. https:\/\/doi.org\/10.1145\/2492248.2492276","journal-title":"SIGSOFT Softw. Eng. Notes"},{"issue":"9","key":"4078_CR14","doi-asserted-by":"publisher","first-page":"1857","DOI":"10.1109\/TSE.2019.2937083 10.1109\/TSE.2019.2937083","volume":"47","author":"Z Wan","year":"2021","unstructured":"Wan Z, Xia X, Lo D, Murphy GC (2021) How does machine learning change software development practices? IEEE Trans Softw Eng 47(9):1857\u20131871. https:\/\/doi.org\/10.1109\/TSE.2019.2937083https:\/\/doi.org\/10.1109\/TSE.2019.2937083","journal-title":"IEEE Trans Softw Eng"},{"key":"4078_CR15","doi-asserted-by":"publisher","unstructured":"Amershi S, Begel A, Bird C, DeLine R, Gall H, Kamar E, Nagappan N, Nushi B, Zimmermann T (2019) Software engineering for machine learning: a case study. In: 2019 IEEE\/ACM 41st international conference on software engineering: software engineering in practice (ICSE-SEIP), pp 291\u2013300. https:\/\/doi.org\/10.1109\/ICSE-SEIP.2019.00042","DOI":"10.1109\/ICSE-SEIP.2019.00042"},{"issue":"8","key":"4078_CR16","doi-asserted-by":"publisher","first-page":"1735","DOI":"10.1162\/neco.1997.9.8.1735","volume":"9","author":"S Hochreiter","year":"1997","unstructured":"Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735\u20131780. https:\/\/doi.org\/10.1162\/neco.1997.9.8.1735https:\/\/doi.org\/10.1162\/neco.1997.9.8.1735","journal-title":"Neural Comput"},{"issue":"11","key":"4078_CR17","doi-asserted-by":"publisher","first-page":"2673","DOI":"10.1109\/78.650093","volume":"45","author":"M Schuster","year":"1997","unstructured":"Schuster M, Paliwal K.K (1997) Bidirectional recurrent neural networks. IEEE Trans Signal Process 45(11):2673\u20132681. https:\/\/doi.org\/10.1109\/78.650093","journal-title":"IEEE Trans Signal Process"},{"issue":"6","key":"4078_CR18","doi-asserted-by":"publisher","first-page":"84","DOI":"10.1145\/3065386","volume":"60","author":"A Krizhevsky","year":"2017","unstructured":"Krizhevsky A, Sutskever I, Hinton G.E (2017) Imagenet classification with deep convolutional neural networks. Commun ACM 60(6):84\u201390. https:\/\/doi.org\/10.1145\/3065386","journal-title":"Commun ACM"},{"key":"4078_CR19","doi-asserted-by":"crossref","unstructured":"Gao H, Xiao J, Yin Y, Liu T, Shi J (2022) A mutually supervised graph attention network for few-shot segmentation: the perspective of fully utilizing limited samples. IEEE Trans Neural Netw Learn Syst :1\u201313","DOI":"10.1109\/TNNLS.2022.3155486"},{"issue":"1","key":"4078_CR20","doi-asserted-by":"publisher","first-page":"336","DOI":"10.1109\/TCSS.2021.3102591","volume":"9","author":"H Gao","year":"2022","unstructured":"Gao H, Xu K, Cao M, Xiao J, Xu Q, Yin Y (2022) The deep features and attention mechanism-based method to dish healthcare under social iot systems: an empirical study with a hand-deep local\u2013global net. IEEE Trans Comput Soc Syst 9(1):336\u2013347","journal-title":"IEEE Trans Comput Soc Syst"},{"issue":"1s","key":"4078_CR21","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3419842","volume":"17","author":"J Xiao","year":"2021","unstructured":"Xiao J, Xu H, Gao H, Bian M, Li Y (2021) A weakly supervised semantic segmentation network by aggregating seed cues: the multi-object proposal generation perspective. ACM Trans Multimed Comput Commun Appl 17(1s):1\u201319","journal-title":"ACM Trans Multimed Comput Commun Appl"},{"issue":"18","key":"4078_CR22","doi-asserted-by":"publisher","first-page":"3790","DOI":"10.3390\/app9183790","volume":"9","author":"MA Rahim","year":"2019","unstructured":"Rahim MA, Islam MR, Shin J (2019) Non-touch sign word recognition based on dynamic hand gesture using hybrid segmentation and cnn feature fusion. Appl Sci 9(18):3790. https:\/\/doi.org\/10.3390\/app9183790","journal-title":"Appl Sci"},{"issue":"4","key":"4078_CR23","doi-asserted-by":"publisher","first-page":"861","DOI":"10.3390\/s19040861","volume":"19","author":"X Ran","year":"2019","unstructured":"Ran X, Shan Z, Fang Y, Lin C (2019) An lstm-based method with attention mechanism for travel time prediction. Sensors 19(4):861. https:\/\/doi.org\/10.3390\/s19040861","journal-title":"Sensors"},{"key":"4078_CR24","doi-asserted-by":"crossref","unstructured":"Gao H, Qiu B, Duran Barroso RJ, Hussain W, Xu Y, Wang X (2022) Tsmae: a novel anomaly detection approach for internet of things time series data using memory-augmented autoencoder. In: IEEE Transactions on Network Science and Engineering, pp 1\u20131","DOI":"10.1109\/TNSE.2022.3163144"},{"key":"4078_CR25","doi-asserted-by":"publisher","first-page":"12929","DOI":"10.1109\/ACCESS.2018.2794765","volume":"6","author":"H Zhao","year":"2018","unstructured":"Zhao H, Sun S, Jin B (2018) Sequential fault diagnosis based on lstm neural network. IEEE Access 6:12929\u201312939. https:\/\/doi.org\/10.1109\/ACCESS.2018.2794765","journal-title":"IEEE Access"},{"key":"4078_CR26","doi-asserted-by":"crossref","unstructured":"Gao H, Huang W, Liu T, Yin Y, Li Y (2022) Ppo2: location privacy-oriented task offloading to edge computing using reinforcement learning for intelligent autonomous transport systems. In: IEEE Transactions on Intelligent Transportation Systems, pp 1\u201314","DOI":"10.1109\/TITS.2022.3169421"},{"key":"4078_CR27","doi-asserted-by":"publisher","first-page":"04014","DOI":"10.1051\/shsconf\/202110204014","volume":"102","author":"MM Rahman","year":"2021","unstructured":"Rahman MM, Kawabayashi S, Watanobe Y (2021) Categorization of frequent errors in solution codes created by novice programmers. SHS Web Conf 102:04014. https:\/\/doi.org\/10.1051\/shsconf\/202110204014","journal-title":"SHS Web Conf"},{"key":"4078_CR28","doi-asserted-by":"publisher","first-page":"14","DOI":"10.1155\/2019\/6230953","volume":"2019","author":"G Fan","year":"2019","unstructured":"Fan G, Diao X, Yu H, Yang K, Chen L, Vitiello A (2019) Software defect prediction via attention-based recurrent neural network. Sci Program 2019:14. https:\/\/doi.org\/10.1155\/2019\/6230953","journal-title":"Sci Program"},{"issue":"2-3","key":"4078_CR29","doi-asserted-by":"publisher","first-page":"78","DOI":"10.1504\/IJCISTUDIES.2021.115424","volume":"10","author":"K Terada","year":"2021","unstructured":"Terada K, Watanobe Y (2021) Code completion for programming education based on deep learning. Int J Comput Intell Stud 10(2-3):78\u201398. https:\/\/doi.org\/10.1504\/IJCISTUDIES.2021.115424","journal-title":"Int J Comput Intell Stud"},{"key":"4078_CR30","doi-asserted-by":"publisher","unstructured":"Ohashi H, Watanobe Y (2019) Convolutional neural network for classification of source codes. In: 2019 IEEE 13th international symposium on embedded multicore\/many-core systems-on-chip (MCSoC), pp 194\u2013200. https:\/\/doi.org\/10.1109\/MCSoC.2019.00035https:\/\/doi.org\/10.1109\/MCSoC.2019.00035","DOI":"10.1109\/MCSoC.2019.00035 10.1109\/MCSoC.2019.00035"},{"key":"4078_CR31","doi-asserted-by":"publisher","first-page":"18","DOI":"10.1155\/2020\/7426461","volume":"2020","author":"MM Rahman","year":"2020","unstructured":"Rahman MM, Watanobe Y, Nakamura K (2020) A neural network based intelligent support model for program code completion. Sci Program 2020:18. https:\/\/doi.org\/10.1155\/2020\/7426461","journal-title":"Sci Program"},{"issue":"2","key":"4078_CR32","doi-asserted-by":"publisher","first-page":"247","DOI":"10.3390\/sym13020247","volume":"13","author":"MM Rahman","year":"2021","unstructured":"Rahman MM, Watanobe Y, Nakamura K (2021) A bidirectional lstm language model for code evaluation and repair. Symmetry 13(2):247. https:\/\/doi.org\/10.3390\/sym13020247","journal-title":"Symmetry"},{"key":"4078_CR33","doi-asserted-by":"publisher","unstructured":"Rahman MM, Watanobe Y, Nakamura K (2020) Evaluation of source codes using bidirectional lstm neural network. In: 2020 3rd IEEE international conference on knowledge innovation and invention (ICKII), pp 140\u2013143. https:\/\/doi.org\/10.1109\/ICKII50300.2020.9318916https:\/\/doi.org\/10.1109\/ICKII50300.2020.9318916","DOI":"10.1109\/ICKII50300.2020.9318916 10.1109\/ICKII50300.2020.9318916"},{"issue":"2","key":"4078_CR34","doi-asserted-by":"publisher","first-page":"277","DOI":"10.1007\/s10489-016-0892-x","volume":"47","author":"R Yera","year":"2017","unstructured":"Yera R, Mart\u00ednez L (2017) A recommendation approach for programming online judges supported by data preprocessing techniques. Appl Intell 47(2):277\u2013290. https:\/\/doi.org\/10.1007\/s10489-016-0892-xhttps:\/\/doi.org\/10.1007\/s10489-016-0892-x","journal-title":"Appl Intell"},{"issue":"1","key":"4078_CR35","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3143560","volume":"51","author":"S Wasik","year":"2018","unstructured":"Wasik S, Antczak M, Badura J, Laskowski A, Sternal T (2018) A survey on online judge systems and their applications. ACM Comput Surv 51(1):1\u201334. https:\/\/doi.org\/10.1145\/3143560","journal-title":"ACM Comput Surv"},{"key":"4078_CR36","unstructured":"Project CodeNet (2021) [Online] Available: https:\/\/github.com\/IBM\/Project_CodeNet. Accessed 10 Jan 2022"},{"key":"4078_CR37","doi-asserted-by":"publisher","unstructured":"Li Y, Choi D, Chung J, Kushman N, Schrittwieser J, Leblond R, Eccles T, Keeling J, Gimeno F, Lago AD, Hubert T, Choy P, d\u2019Autume CdM, Babuschkin I, Chen X, Huang P-S, Welbl J, Gowal S, Cherepanov A, Molloy J, Mankowitz D.J, Robson ES, Kohli P, de Freitas N, Kavukcuoglu K, Vinyals O (2022) Competition-Level Code Generation with AlphaCode. https:\/\/doi.org\/10.48550\/ARXIV.2203.07814","DOI":"10.48550\/ARXIV.2203.07814"},{"key":"4078_CR38","doi-asserted-by":"publisher","unstructured":"Lu S, Guo D, Ren S, Huang J, Svyatkovskiy A, Blanco A, Clement C, Drain D, Jiang D, Tang D, Li G, Zhou L, Shou L, Zhou L, Tufano M, Gong M, Zhou M, Duan N, Sundaresan N, Deng SK, Fu S, Liu S (2021) CodeXGLUE: a machine learning benchmark dataset for code understanding and generation. https:\/\/doi.org\/10.48550\/ARXIV.2102.04664","DOI":"10.48550\/ARXIV.2102.04664"},{"issue":"11","key":"4078_CR39","doi-asserted-by":"publisher","first-page":"4489","DOI":"10.3390\/su12114489","volume":"12","author":"T-L Chen","year":"2020","unstructured":"Chen T-L, Hsiao T-C, Kang T-C, Wu T-Y, Chen C-C (2020) Learning programming language in higher education for sustainable development: point-earning bidding method. Sustainability 12(11):4489. https:\/\/doi.org\/10.3390\/su12114489","journal-title":"Sustainability"},{"key":"4078_CR40","doi-asserted-by":"publisher","first-page":"26186","DOI":"10.1109\/ACCESS.2022.3157288","volume":"10","author":"MM Rahman","year":"2022","unstructured":"Rahman MM, Watanobe Y, Matsumoto T, Kiran RU, Nakamura K (2022) Educational data mining to support programming learning using problem-solving data. IEEE Access 10:26186\u201326202. https:\/\/doi.org\/10.1109\/ACCESS.2022.3157288","journal-title":"IEEE Access"},{"key":"4078_CR41","doi-asserted-by":"publisher","unstructured":"Sun Q, Wu J, Liu K (2019) How are students\u2019 programming skills developed: an empirical study in an object-oriented course. In: Proceedings of the ACM turing celebration conference - China. ACM TURC \u201919. https:\/\/doi.org\/10.1145\/3321408.3322858","DOI":"10.1145\/3321408.3322858"},{"issue":"1","key":"4078_CR42","doi-asserted-by":"publisher","first-page":"1","DOI":"10.1145\/3077618","volume":"18","author":"Y Qian","year":"2017","unstructured":"Qian Y, Lehman J (2017) Students\u2019 misconceptions and other difficulties in introductory programming: a literature review. ACM Trans Comput Educ 18(1):1\u201324. https:\/\/doi.org\/10.1145\/3077618","journal-title":"ACM Trans Comput Educ"},{"issue":"1","key":"4078_CR43","doi-asserted-by":"publisher","first-page":"33","DOI":"10.4018\/IJOPCD.2017010103","volume":"7","author":"BS Xia","year":"2017","unstructured":"Xia BS (2017) A pedagogical review of programming education research: what have we learned. Int J Online Pedagog Course Des 7(1):33\u201342. https:\/\/doi.org\/10.4018\/IJOPCD.2017010103","journal-title":"Int J Online Pedagog Course Des"},{"issue":"6245","key":"4078_CR44","doi-asserted-by":"publisher","first-page":"255","DOI":"10.1126\/science.aaa8415","volume":"349","author":"MI Jordan","year":"2015","unstructured":"Jordan MI, Mitchell TM (2015) Machine learning: trends, perspectives, and prospects. Science 349(6245):255\u2013260. https:\/\/doi.org\/10.1126\/science.aaa8415","journal-title":"Science"},{"key":"4078_CR45","doi-asserted-by":"publisher","unstructured":"Salvaris M, Dean D, Tok WH (2018) Microsoft AI platform. Apress, pp 79\u201398. https:\/\/doi.org\/10.1007\/978-1-4842-3679-6_4","DOI":"10.1007\/978-1-4842-3679-6_4"},{"key":"4078_CR46","first-page":"5","volume":"41","author":"S Schelter","year":"2018","unstructured":"Schelter S, Biessmann F, Januschowski T, Salinas D, Seufert S, Szarvas G (2018) On challenges in machine learning model management. IEEE Data Eng Bull 41:5\u201315","journal-title":"IEEE Data Eng Bull"},{"key":"4078_CR47","unstructured":"Martin Z (2019) Rules of machine learning: best practices for ML engineering https:\/\/developers.google.com\/machine-learning\/guides\/rules-of-ml\/. Accessed 25 Dec 2021"},{"key":"4078_CR48","doi-asserted-by":"publisher","first-page":"120","DOI":"10.1145\/3238147.3238202","volume-title":"DeepGauge: multi-granularity testing criteria for deep learning systems","author":"L Ma","year":"2018","unstructured":"Ma L, Juefei-Xu F, Zhang F, Sun J, Xue M, Li B, Chen C, Su T, Li L, Liu Y, Zhao J, Wang Y (2018) DeepGauge: multi-granularity testing criteria for deep learning systems. Association for Computing Machinery, New York, pp 120\u2013131. https:\/\/doi.org\/10.1145\/3238147.3238202"},{"issue":"11","key":"4078_CR49","doi-asserted-by":"publisher","first-page":"137","DOI":"10.1145\/3361566","volume":"62","author":"K Pei","year":"2019","unstructured":"Pei K, Cao Y, Yang J, Jana S (2019) Deepxplore: automated whitebox testing of deep learning systems. Commun ACM 62(11):137\u2013145. https:\/\/doi.org\/10.1145\/336.1566","journal-title":"Commun ACM"},{"issue":"4","key":"4078_CR50","doi-asserted-by":"publisher","first-page":"544","DOI":"10.1016\/j.jss.2010.11.920","volume":"84","author":"X Xie","year":"2011","unstructured":"Xie X, Ho JWK, Murphy C, Kaiser G, Xu B, Chen TY (2011) Testing and validating machine learning classifiers by metamorphic testing. J Syst Softw 84(4):544\u2013558. https:\/\/doi.org\/10.1016\/j.jss.2010.11.920","journal-title":"J Syst Softw"},{"key":"4078_CR51","doi-asserted-by":"publisher","unstructured":"Ma S, Liu Y, Lee W-C, Zhang X, Grama A (2018) Mode: Automated neural network model debugging via state differential analysis and input selection. In: Proceedings of the 2018 26th ACM joint meeting on european software engineering conference and symposium on the foundations of software engineering. ESEC\/FSE 2018, Association for Computing Machinery, pp 175\u2013186. https:\/\/doi.org\/10.1145\/3236024.3236082","DOI":"10.1145\/3236024.3236082"},{"key":"4078_CR52","doi-asserted-by":"publisher","unstructured":"Mou L, Li G, Zhang L, Wang T, Jin Z (2016) Convolutional neural networks over tree structures for programming language processing. In: Proceedings of the Thirtieth AAAI conference on artificial intelligence. AAAI\u201916, pp 1287\u20131293. https:\/\/doi.org\/10.5555\/3015812.3016002","DOI":"10.5555\/3015812.3016002"},{"key":"4078_CR53","doi-asserted-by":"publisher","unstructured":"Wan Y, Shu J, Sui Y, Xu G, Zhao Z, Wu J, Yu P (2019) Multi-modal attention network learning for semantic source code retrieval. In: 2019 34th IEEE\/ACM international conference on automated software engineering (ASE), pp 13\u201325. https:\/\/doi.org\/10.1109\/ASE.2019.00012","DOI":"10.1109\/ASE.2019.00012"},{"key":"4078_CR54","doi-asserted-by":"crossref","unstructured":"Hindle A, Barr ET, Su Z, Gabel M, Devanbu P (2012) On the naturalness of software. In: Proceedings of the 34th international conference on software engineering. ICSE \u201912, pp 837\u2013847","DOI":"10.1109\/ICSE.2012.6227135"},{"issue":"6","key":"4078_CR55","doi-asserted-by":"publisher","first-page":"419","DOI":"10.1145\/2666356.2594321","volume":"49","author":"V Raychev","year":"2014","unstructured":"Raychev V, Vechev M, Yahav E (2014) Code completion with statistical language models. ACM SIGPLAN Notices 49(6):419\u2013428. https:\/\/doi.org\/10.1145\/2666356.2594321","journal-title":"ACM SIGPLAN Notices"},{"key":"4078_CR56","unstructured":"Bui N, Jiang L, Yu Y Cross-language learning for program classification using bilateral tree-based convolutional neural networks. https:\/\/www.aaai.org\/ocs\/index.php\/WS\/AAAIW18\/paper\/view\/17338\/15660"},{"key":"4078_CR57","doi-asserted-by":"publisher","first-page":"87857","DOI":"10.1109\/ACCESS.2021.3063475","volume":"9","author":"M Lu","year":"2021","unstructured":"Lu M, Wang Y, Tan D, Zhao L (2021) Student program classification using gated graph attention neural network. IEEE Access 9:87857\u201387868. https:\/\/doi.org\/10.1109\/ACCESS.2021.3063475","journal-title":"IEEE Access"},{"key":"4078_CR58","doi-asserted-by":"publisher","unstructured":"Ugurel S, Krovetz R, Giles CL (2002) What\u2019s the code? automatic classification of source code archives. In: Proceedings of the eighth ACM SIGKDD international conference on knowledge discovery and data mining. KDD \u201902, ACM, pp 632\u2013638. https:\/\/doi.org\/10.1145\/775047.775141","DOI":"10.1145\/775047.775141"},{"key":"4078_CR59","doi-asserted-by":"publisher","unstructured":"Tian K, Revelle M, Poshyvanyk D (2009) Using latent dirichlet allocation for automatic categorization of software. In: 2009 6th IEEE international working conference on mining software repositories, pp 163\u2013166. https:\/\/doi.org\/10.1109\/MSR.2009.5069496","DOI":"10.1109\/MSR.2009.5069496"},{"key":"4078_CR60","doi-asserted-by":"publisher","unstructured":"Alreshedy K, Dharmaretnam D, German DM, Srinivasan V, Gulliver TA (2018) Scc: automatic classification of code snippets. In: 2018 IEEE 18th international working conference on source code analysis and manipulation (SCAM), pp 203\u2013208. https:\/\/doi.org\/10.1109\/SCAM.2018.00031","DOI":"10.1109\/SCAM.2018.00031"},{"key":"4078_CR61","doi-asserted-by":"publisher","unstructured":"Reyes J, Ram\u00edrez D, Paciello J (2016) Automatic classification of source code archives by programming language: a deep learning approach. In: 2016 international conference on computational science and computational intelligence (CSCI), pp 514\u2013519. https:\/\/doi.org\/10.1109\/CSCI.2016.0103","DOI":"10.1109\/CSCI.2016.0103"},{"key":"4078_CR62","doi-asserted-by":"publisher","unstructured":"Gilda S (2017) Source code classification using neural networks. In: 2017 14th international joint conference on computer science and software engineering (JCSSE), pp 1\u20136. https:\/\/doi.org\/10.1109\/JCSSE.2017.8025917","DOI":"10.1109\/JCSSE.2017.8025917"},{"key":"4078_CR63","doi-asserted-by":"publisher","unstructured":"Shalaby M, Mehrez T, El Mougy A, Abdulnasser K, Al-Safty A (2017) Automatic algorithm recognition of source-code using machine learning. In: 2017 16th IEEE international conference on machine learning and applications (ICMLA), pp 170\u2013177. https:\/\/doi.org\/10.1109\/ICMLA.2017.00033","DOI":"10.1109\/ICMLA.2017.00033"},{"key":"4078_CR64","doi-asserted-by":"publisher","unstructured":"Taherkhani A (2010) Recognizing sorting algorithms with the c4.5 decision tree classifier. In: 2010 IEEE 18th International Conference on Program Comprehension, pp 72\u201375. https:\/\/doi.org\/10.1109\/ICPC.2010.11","DOI":"10.1109\/ICPC.2010.11"},{"key":"4078_CR65","doi-asserted-by":"publisher","unstructured":"LeClair A, Eberhart Z, McMillan C (2018) Adapting neural text classification for improved software categorization. In: 2018 IEEE international conference on software maintenance and evolution (ICSME), pp 461\u2013472. https:\/\/doi.org\/10.1109\/ICSME.2018.00056","DOI":"10.1109\/ICSME.2018.00056"},{"key":"4078_CR66","doi-asserted-by":"publisher","unstructured":"Xu A, Dai T, Chen H, Ming Z, Li W (2018) Vulnerability detection for source code using contextual lstm. In: 2018 5th international conference on systems and informatics (ICSAI), pp 1225\u20131230. https:\/\/doi.org\/10.1109\/ICSAI.2018.8599360","DOI":"10.1109\/ICSAI.2018.8599360"},{"key":"4078_CR67","doi-asserted-by":"publisher","unstructured":"Kim Y (2014) Convolutional neural networks for sentence classification. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), Association for Computational Linguistics, pp 1746\u20131751. https:\/\/doi.org\/10.3115\/v1\/D14-1181","DOI":"10.3115\/v1\/D14-1181"},{"key":"4078_CR68","doi-asserted-by":"publisher","first-page":"157158","DOI":"10.1109\/ACCESS.2019.2949483","volume":"7","author":"S Dey","year":"2019","unstructured":"Dey S, Singh AK, Prasad DK, Mcdonald-Maier KD (2019) Socodecnn: program source code for visual cnn classification using computer vision methodology. IEEE Access 7:157158\u2013157172. https:\/\/doi.org\/10.1109\/ACCESS.2019.2949483","journal-title":"IEEE Access"},{"key":"4078_CR69","unstructured":"Watanobe Y (2018) Aizu online judge available: https:\/\/onlinejudge.u-aizu.ac.jp\/. Accessed 1 Feb 2022"},{"key":"4078_CR70","unstructured":"Aizu Online Judge (2004) Developers site (API) Available: http:\/\/developers.u-aizu.ac.jp\/index. Accessed 1 Feb 2022"},{"key":"4078_CR71","unstructured":"Puri R, Kung DS, Janssen G, Zhang W, Domeniconi G, Zolotov V, Dolby J, Chen J, Choudhury MR, Decker L, Thost V, Buratti L, Pujar S, Finkler U (2021) Project codenet: A large-scale ai for code dataset for learning a diversity of coding tasks. arXiv:2105.12655"},{"key":"4078_CR72","doi-asserted-by":"publisher","first-page":"748","DOI":"10.1016\/j.asoc.2018.09.010","volume":"73","author":"SF Chevtchenko","year":"2018","unstructured":"Chevtchenko SF, Vale RF, Macario V, Cordeiro FR (2018) A convolutional neural network with feature fusion for real-time hand posture recognition. Appl Soft Comput 73:748\u2013766. https:\/\/doi.org\/10.1016\/j.asoc.2018.09.010","journal-title":"Appl Soft Comput"},{"issue":"5","key":"4078_CR73","doi-asserted-by":"publisher","first-page":"1671","DOI":"10.4208\/cicp.OA-2020-0165","volume":"28","author":"L Lu","year":"2020","unstructured":"Lu L, Shin Y, Su Y, Em Karniadakis G (2020) Dying relu and initialization: theory and numerical examples. Commun Comput 28(5):1671\u20131706. https:\/\/doi.org\/10.4208\/cicp.OA-2020-0165","journal-title":"Commun Comput"},{"key":"4078_CR74","doi-asserted-by":"publisher","unstructured":"Rahman MM, Watanobe Y, Kiran RU, Kabir R (2021) A stacked bidirectional lstm model for classifying source codes built in mpls. In: Machine learning and principles and practice of knowledge discovery in databases, pp 75\u201389. https:\/\/doi.org\/10.1007\/978-3-030-93733-1_5","DOI":"10.1007\/978-3-030-93733-1_5"},{"key":"4078_CR75","doi-asserted-by":"publisher","unstructured":"Borges O, Couto J, Ruiz D, Prikladnicki R (2020) How machine learning has been applied in software engineering?. In: Proceedings of the 22nd international conference on enterprise information systems - volume 2: ICEIS, pp 306\u2013313. https:\/\/doi.org\/10.5220\/0009417703060313","DOI":"10.5220\/0009417703060313"}],"container-title":["Applied Intelligence"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10489-022-04078-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10489-022-04078-y\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10489-022-04078-y.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2023,5,20]],"date-time":"2023-05-20T10:35:13Z","timestamp":1684578913000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10489-022-04078-y"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2022,9,23]]},"references-count":75,"journal-issue":{"issue":"10","published-print":{"date-parts":[[2023,5]]}},"alternative-id":["4078"],"URL":"https:\/\/doi.org\/10.1007\/s10489-022-04078-y","relation":{},"ISSN":["0924-669X","1573-7497"],"issn-type":[{"value":"0924-669X","type":"print"},{"value":"1573-7497","type":"electronic"}],"subject":[],"published":{"date-parts":[[2022,9,23]]},"assertion":[{"value":"8 August 2022","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"23 September 2022","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}},{"order":1,"name":"Ethics","group":{"name":"EthicsHeading","label":"Declarations"}},{"value":"The authors declare that they have no conflicts of interest.","order":2,"name":"Ethics","group":{"name":"EthicsHeading","label":"<!--Emphasis Type='Bold' removed-->Conflict of Interests"}}]}}