{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,1,13]],"date-time":"2026-01-13T21:43:28Z","timestamp":1768340608006,"version":"3.49.0"},"reference-count":73,"publisher":"Springer Science and Business Media LLC","issue":"4","license":[{"start":{"date-parts":[[2021,5,14]],"date-time":"2021-05-14T00:00:00Z","timestamp":1620950400000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"},{"start":{"date-parts":[[2021,5,14]],"date-time":"2021-05-14T00:00:00Z","timestamp":1620950400000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/creativecommons.org\/licenses\/by\/4.0"}],"funder":[{"DOI":"10.13039\/100010669","name":"H2020 LEIT Information and Communication Technologies","doi-asserted-by":"publisher","award":["732223"],"award-info":[{"award-number":["732223"]}],"id":[{"id":"10.13039\/100010669","id-type":"DOI","asserted-by":"publisher"}]},{"DOI":"10.13039\/501100006256","name":"Universit\u00e0 degli Studi dell\u2019Aquila","doi-asserted-by":"crossref","id":[{"id":"10.13039\/501100006256","id-type":"DOI","asserted-by":"crossref"}]}],"content-domain":{"domain":["link.springer.com"],"crossmark-restriction":false},"short-container-title":["Empir Software Eng"],"published-print":{"date-parts":[[2021,7]]},"abstract":"<jats:title>Abstract<\/jats:title><jats:p>To perform their daily tasks, developers intensively make use of existing resources by consulting open source software (OSS) repositories. Such platforms contain rich data sources, e.g., code snippets, documentations, and user discussions, that can be useful for supporting development activities. Over the last decades, several techniques and tools have been promoted to provide developers with innovative features, aiming to bring in improvements in terms of development effort, cost savings, and productivity. In the context of the EU H2020 CROSSMINER project, a set of recommendation systems has been conceived to assist software programmers in different phases of the development process. The systems provide developers with various artifacts, such as third-party libraries, documentation about how to use the APIs being adopted, or relevant API function calls. To develop such recommendations, various technical choices have been made to overcome issues related to several aspects including the lack of baselines, limited data availability, decisions about the performance measures, and evaluation approaches. This paper is an experience report to present the knowledge pertinent to the set of recommendation systems developed through the CROSSMINER project. We explain in detail the challenges we had to deal with, together with the related lessons learned when developing and evaluating these systems. Our aim is to provide the research community with concrete takeaway messages that are expected to be useful for those who want to develop or customize their own recommendation systems. The reported experiences can facilitate interesting discussions and research work, which in the end contribute to the advancement of recommendation systems applied to solve different issues in Software Engineering.<\/jats:p>","DOI":"10.1007\/s10664-021-09963-7","type":"journal-article","created":{"date-parts":[[2021,5,14]],"date-time":"2021-05-14T17:04:41Z","timestamp":1621011881000},"update-policy":"https:\/\/doi.org\/10.1007\/springer_crossmark_policy","source":"Crossref","is-referenced-by-count":36,"title":["Development of recommendation systems for software engineering: the CROSSMINER experience"],"prefix":"10.1007","volume":"26","author":[{"given":"Juri","family":"Di Rocco","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"ORCID":"https:\/\/orcid.org\/0000-0002-5077-6793","authenticated-orcid":false,"given":"Davide","family":"Di Ruscio","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Claudio","family":"Di Sipio","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Phuong T.","family":"Nguyen","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Riccardo","family":"Rubei","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"297","published-online":{"date-parts":[[2021,5,14]]},"reference":[{"key":"9963_CR1","doi-asserted-by":"crossref","unstructured":"Alreshedy K., Dharmaretnam D., German D.M., Srinivasan V., Gulliver T.A. (2018) SCC: automatic classification of code snippets. CoRR arXiv:1809.07945","DOI":"10.1109\/SCAM.2018.00031"},{"key":"9963_CR2","doi-asserted-by":"crossref","unstructured":"Basten B., Hills M., Klint P., Landman D., Shahi A., Steindorfer M.J., Vinju J.J. (2015) M3: a general model for code analytics in rascal. In: 2015 IEEE 1st international workshop on software analytics (SWAN), pp 25\u201328","DOI":"10.1109\/SWAN.2015.7070485"},{"key":"9963_CR3","doi-asserted-by":"publisher","first-page":"142","DOI":"10.1016\/j.ins.2012.09.039","volume":"221","author":"A Bellog\u00edn","year":"2013","unstructured":"Bellog\u00edn A., Cantador I., Castells P. (2013) A comparative study of heterogeneous item recommendations in social systems. Information Science 221:142\u2013169. https:\/\/doi.org\/10.1016\/j.ins.2012.09.039","journal-title":"Information Science"},{"key":"9963_CR4","doi-asserted-by":"publisher","first-page":"647","DOI":"10.2307\/20453570","volume":"46","author":"VD Blondel","year":"2004","unstructured":"Blondel V.D., Gajardo A., Heymans M., Senellart P., Dooren P.V. (2004) A measure of similarity between graph vertices: applications to synonym extraction and web searching. SIAM Review 46:647\u2013666. https:\/\/doi.org\/10.2307\/20453570","journal-title":"SIAM Review"},{"key":"9963_CR5","doi-asserted-by":"publisher","first-page":"109","DOI":"10.1016\/j.knosys.2013.03.012","volume":"46","author":"J Bobadilla","year":"2013","unstructured":"Bobadilla J., Ortega F., Hernando A., Guti\u00e9rrez A. (2013) Recommender systems survey. Knowledge-Based Systems 46:109\u2013132","journal-title":"Knowledge-Based Systems"},{"key":"9963_CR6","doi-asserted-by":"publisher","first-page":"112","DOI":"10.1016\/j.jss.2018.09.016","volume":"146","author":"H Borges","year":"2018","unstructured":"Borges H., Valente M.T. (2018) What\u2019s in a GitHub star? Understanding repository starring practices in a social coding platform. Journal of Systems and Software 146:112\u2013129. arXiv:1811.07643. https:\/\/doi.org\/10.1016\/j.jss.2018.09.016","journal-title":"Journal of Systems and Software"},{"key":"9963_CR7","doi-asserted-by":"crossref","unstructured":"Bruch M., Sch\u00e4fer T., Mezini M. (2008) On evaluating recommender systems for API usages. In: Proceedings of the 2008 international workshop on recommendation systems for software engineering, RSSE \u201908. ACM, New York, pp 16\u201320","DOI":"10.1145\/1454247.1454254"},{"key":"9963_CR8","doi-asserted-by":"crossref","unstructured":"Castells P, Vargas S, Wang J (2011) Novelty and diversity metrics for recommender systems: choice, discovery and relevance. In: International workshop on diversity in document retrieval (DDR 2011) at the 33rd European conference on information retrieval (ECIR 2011). Dublin, Ireland. http:\/\/ir.ii.uam.es\/rim3\/publications\/ddr11.pdf","DOI":"10.1145\/2043932.2043955"},{"key":"9963_CR9","doi-asserted-by":"publisher","first-page":"7173","DOI":"10.1109\/ACCESS.2017.2682323","volume":"5","author":"V Cosentino","year":"2017","unstructured":"Cosentino V., C\u00e1novas Izquierdo J.L., Cabot J. (2017) A systematic mapping study of software development with github. IEEE Access 5:7173\u20137192. https:\/\/doi.org\/10.1109\/ACCESS.2017.2682323","journal-title":"IEEE Access"},{"key":"9963_CR10","doi-asserted-by":"publisher","unstructured":"Czarnecki K. (2002) Domain engineering. pp. 433\u2013444. American Cancer Society. https:\/\/doi.org\/10.1002\/0471028959.sof095. https:\/\/onlinelibrary.wiley.com\/doi\/abs\/10.1002\/0471028959.sof095","DOI":"10.1002\/0471028959.sof095"},{"key":"9963_CR11","doi-asserted-by":"crossref","unstructured":"Dagenais B., Ossher H., Bellamy R.K.E., Robillard M.P., de Vries J.P. (2010) Moving into a new software project landscape. In: Proceedings of the 32nd ACM\/IEEE international conference on software engineering - volume 1, ICSE \u201910. ACM, New York, pp 275\u2013284","DOI":"10.1145\/1806799.1806842"},{"key":"9963_CR12","doi-asserted-by":"crossref","unstructured":"Davis J., Goadrich M. (2006) The relationship between precision-recall and ROC curves. In: Proceedings of the 23rd international conference on machine learning, ICML \u201906. ACM, New York, pp 233\u2013240","DOI":"10.1145\/1143844.1143874"},{"key":"9963_CR13","unstructured":"Di Cosmo R., Zacchiroli S. (2017) Software heritage: why and how to preserve software source code. In: 14th international conference on digital preservation. Kyoto, pp 1\u201310"},{"key":"9963_CR14","doi-asserted-by":"crossref","unstructured":"Di Noia T., Mirizzi R., Ostuni V.C., Romito D., Zanker M. (2012) Linked open data to support content-based recommender systems. In: Proceedings of the 8th international conference on semantic systems, I-semantics \u201912. ACM, New York, pp 1\u20138","DOI":"10.1145\/2362499.2362501"},{"key":"9963_CR15","doi-asserted-by":"publisher","unstructured":"Di Rocco J., Di Ruscio D., Di Sipio C., Nguyen P., Rubei R. (2020) Topfilter: an approach to recommend relevant github topics. In: Proceedings of the 14th ACM \/ IEEE international symposium on empirical software engineering and measurement (ESEM), ESEM \u201920. https:\/\/doi.org\/10.1145\/3382494.3410690. Association for Computing Machinery, New York","DOI":"10.1145\/3382494.3410690"},{"key":"9963_CR16","doi-asserted-by":"publisher","unstructured":"Di Sipio C., Di Ruscio D., Nguyen P.T. (2020) Democratizing the development of recommender systems by means of low-code platforms. In: Proceedings of the 23rd ACM\/IEEE international conference on model driven engineering languages and systems: companion proceedings, MODELS \u201920. https:\/\/doi.org\/10.1145\/3417990.3420202. Association for Computing Machinery, New York","DOI":"10.1145\/3417990.3420202"},{"key":"9963_CR17","doi-asserted-by":"publisher","unstructured":"Di Sipio C., Rubei R., Di Ruscio D., Nguyen P.T. (2020) Using a multinomial na\u00efve bayesian (MNB) network to automatically recommend topics for github repositories. In: Proceedings of the 24th international conference on evaluation and assessment in software engineering, EASE2020, Trondheim, Norway, April 15-17, 2020, EASE\u201920. https:\/\/doi.org\/10.1145\/3383219.3383227. ACM, pp 24\u201334","DOI":"10.1145\/3383219.3383227"},{"key":"9963_CR18","doi-asserted-by":"crossref","unstructured":"Fowkes J., Sutton C. (2016) Parameter-free probabilistic API mining across GitHub. In: Proceedings of the 2016 24th ACM SIGSOFT international symposium on foundations of software engineering, FSE 2016. ACM, New York, pp 254\u2013265","DOI":"10.1145\/2950290.2950319"},{"key":"9963_CR19","unstructured":"Ganesan K. (2017) Topic suggestions for millions of repositories - The GitHub Blog. https:\/\/github.blog\/2017-07-31-topics\/"},{"key":"9963_CR20","unstructured":"Garg P.K., Kawaguchi S., Matsushita M., Inoue K. (2004) MUDABLue: an automatic categorization system for open source repositories. In: 2013 20th Asia-Pacific software engineering conference (APSEC), pp 184\u2013193"},{"key":"9963_CR21","doi-asserted-by":"publisher","first-page":"101","DOI":"10.1016\/j.jss.2015.11.036","volume":"113","author":"M Gasparic","year":"2016","unstructured":"Gasparic M., Janes A. (2016) What recommendation systems for software engineering recommend: a systematic literature review. Journal of Systems and Software 113:101\u2013113. https:\/\/doi.org\/10.1016\/j.jss.2015.11.036. https:\/\/linkinghub.elsevier.com\/retrieve\/pii\/S0164121215002605","journal-title":"Journal of Systems and Software"},{"key":"9963_CR22","doi-asserted-by":"publisher","unstructured":"Ge M., Delgado-Battenfeld C., Jannach D. (2010) Beyond accuracy: evaluating recommender systems by coverage and serendipity. In: Proceedings of the fourth ACM conference on recommender systems, RecSys \u201910. https:\/\/doi.org\/10.1145\/1864708.1864761. Association for Computing Machinery, New York, pp 257\u2013260","DOI":"10.1145\/1864708.1864761"},{"issue":"1","key":"9963_CR23","doi-asserted-by":"publisher","first-page":"26","DOI":"10.1057\/palgrave.jt.5740031","volume":"10","author":"S Ghose","year":"2001","unstructured":"Ghose S., Lowengart O. (2001) Taste tests: impacts of consumer perceptions and preferences on brand positioning strategies. Journal of Targeting, Measurement and Analysis for Marketing 10(1):26\u201341","journal-title":"Journal of Targeting, Measurement and Analysis for Marketing"},{"issue":"4","key":"9963_CR24","first-page":"13:1","volume":"6","author":"CA Gomez-Uribe","year":"2015","unstructured":"Gomez-Uribe C.A., Hunt N. (2015) The netflix recommender system: algorithms, business value, and innovation. ACM Transactions on Management Information Systems 6(4):13:1\u201313:19","journal-title":"ACM Transactions on Management Information Systems"},{"key":"9963_CR25","doi-asserted-by":"crossref","unstructured":"Gousios G. (2013) The ghtorrent dataset and tool suite. In: Proceedings of the 10th working conference on mining software repositories, MSR \u201913. http:\/\/dl.acm.org\/citation.cfm?id=2487085.2487132. IEEE Press, Piscataway, pp 233\u2013236","DOI":"10.1109\/MSR.2013.6624034"},{"key":"9963_CR26","doi-asserted-by":"crossref","unstructured":"Happel H.J., Maalej W. (2008) Potentials and challenges of recommendation systems for software development. In: Proceedings of the 2008 international workshop on Recommendation systems for software engineering - RSSE \u201908. ACM Press, Atlanta, Georgia, p 11","DOI":"10.1145\/1454247.1454251"},{"issue":"5","key":"9963_CR27","doi-asserted-by":"publisher","first-page":"237","DOI":"10.1145\/1095430.1081744","volume":"30","author":"R Holmes","year":"2005","unstructured":"Holmes R., Walker R.J., Murphy G.C. (2005) Strathcona example recommendation tool. SIGSOFT Softw. Eng. Notes 30 (5):237\u2013240. https:\/\/doi.org\/10.1145\/1095430.1081744","journal-title":"SIGSOFT Softw. Eng. Notes"},{"issue":"3","key":"9963_CR28","doi-asserted-by":"publisher","first-page":"261","DOI":"10.1016\/j.eij.2015.06.005","volume":"16","author":"F Isinkaye","year":"2015","unstructured":"Isinkaye F., Folajimi Y., Ojokoh B. (2015) Recommendation systems: principles, methods and evaluation. Egyptian Informatics Journal 16(3):261\u2013273","journal-title":"Egyptian Informatics Journal"},{"issue":"1","key":"9963_CR29","doi-asserted-by":"publisher","first-page":"547","DOI":"10.1007\/s10664-016-9436-6","volume":"22","author":"J Jiang","year":"2017","unstructured":"Jiang J., Lo D., He J., Xia X., Kochhar P.S., Zhang L. (2017) Why and how developers fork what from whom in GitHub. Empirical Software Engineering 22(1):547\u2013578. https:\/\/doi.org\/10.1007\/s10664-016-9436-6","journal-title":"Empirical Software Engineering"},{"key":"9963_CR30","unstructured":"Karlsson E.A. (ed) (1995) Software reuse: a holistic approach. Wiley, New York"},{"issue":"1\/2","key":"9963_CR31","doi-asserted-by":"publisher","first-page":"81","DOI":"10.2307\/2332226","volume":"30","author":"MG Kendall","year":"1938","unstructured":"Kendall M.G. (1938) A new measure of rank correlation. Biometrika 30(1\/2):81\u201393. http:\/\/www.jstor.org\/stable\/2332226","journal-title":"Biometrika"},{"key":"9963_CR32","doi-asserted-by":"crossref","unstructured":"Kibriya A.M., Frank E., Pfahringer B., Holmes G. (2005) Multinomial naive bayes for text categorization revisited. In: Webb G.I., Yu X. (eds) AI 2004: advances in artificial intelligence. Springer, Berlin, pp 488\u2013499","DOI":"10.1007\/978-3-540-30549-1_43"},{"key":"9963_CR33","unstructured":"LASER (2015) LASER: software engineering: international summer schools, LASER 2013-2014, Elba, Italy: revised tutorial lectures. No. 8987 in Lecture notes in computer science Programming and software engineering. Springer, Cham"},{"issue":"1","key":"9963_CR34","doi-asserted-by":"publisher","first-page":"76","DOI":"10.1109\/MIC.2003.1167344","volume":"7","author":"G Linden","year":"2003","unstructured":"Linden G., Smith B., York J. (2003) Amazon.com recommendations: item-to-item collaborative filtering. IEEE Internet Computing 7(1):76\u201380","journal-title":"IEEE Internet Computing"},{"key":"9963_CR35","doi-asserted-by":"crossref","unstructured":"Lv F., Zhang H., Lou J.G., Wang S., Zhang D., Zhao J. (2015) Codehow: effective code search based on API understanding and extended boolean model (E). In: 30th IEEE\/ACM international conference on automated software engineering, ASE 2015, Lincoln, NE, USA, November 9-13, 2015, pp 260\u2013270","DOI":"10.1109\/ASE.2015.42"},{"key":"9963_CR36","unstructured":"Maki S., Kpodjedo S., Boussaidi G.E. (2015) Context extraction in recommendation systems in software engineering: a preliminary survey. In: CASCON \u201915, Markham, Canada. IBM Corp., USA, pp 151\u2013160"},{"key":"9963_CR37","doi-asserted-by":"crossref","unstructured":"McMillan C., Grechanik M., Poshyvanyk D. (2012) Detecting similar software applications. In: Proceedings of the 34th international conference on software engineering, ICSE \u201912. IEEE Press, Piscataway, pp 364\u2013374","DOI":"10.1109\/ICSE.2012.6227178"},{"key":"9963_CR38","doi-asserted-by":"crossref","unstructured":"McMillan C., Poshyvanyk D., Grechanik M. (2010) Recommending source code examples via API call usages and documentation. In: Proceedings of the 2Nd international workshop on recommendation systems for software engineering, RSSE \u201910. ACM, New York, pp 21\u201325","DOI":"10.1145\/1808920.1808925"},{"key":"9963_CR39","doi-asserted-by":"crossref","unstructured":"Moreno L., Bavota G., Di Penta M., Oliveto R., Marcus A. (2015) How can I use this method?. In: 37th international conference on software engineering. IEEE, Piscataway, pp 880\u2013890","DOI":"10.1109\/ICSE.2015.98"},{"key":"9963_CR40","doi-asserted-by":"publisher","unstructured":"Nguyen P., Tomeo P., Di Noia T., Di Sciascio E. (2015) An evaluation of SimRank and personalized PageRank to build a recommender system for the web of data. In: Proceedings of the 24th international conference on world wide web, WWW \u201915 companion. https:\/\/doi.org\/10.1145\/2740908.2742141. Association for Computing Machinery, New York, pp 477\u20131482","DOI":"10.1145\/2740908.2742141"},{"key":"9963_CR41","unstructured":"Nguyen P.T., Di Rocco J., Di Ruscio D. (2018) Mining software repositories to support OSS developers: a recommender systems approach. In: Proceedings of the 9th italian information retrieval workshop, Rome, Italy, May, 28-30, 2018"},{"key":"9963_CR42","doi-asserted-by":"publisher","unstructured":"Nguyen P.T., Di Rocco J., Di Ruscio D., Di Penta M. (2019) CrossRec: supporting software developers by recommending third-party libraries. Journal of Systems and Software, 110460. https:\/\/doi.org\/10.1016\/j.jss.2019.110460. http:\/\/www.sciencedirect.com\/science\/article\/pii\/S0164121219302341","DOI":"10.1016\/j.jss.2019.110460"},{"key":"9963_CR43","doi-asserted-by":"crossref","unstructured":"Nguyen P.T., Di Rocco J., Di Ruscio D., Ochoa L., Degueule T., Di Penta M. (2019) FOCUS: A recommender system for mining API function calls and usage patterns. In: Proceedings of the 41st international conference on software engineering, ICSE \u201919. IEEE Press, Piscataway, pp 1050\u20131060","DOI":"10.1109\/ICSE.2019.00109"},{"key":"9963_CR44","doi-asserted-by":"publisher","unstructured":"Nguyen P.T., Di Rocco J., Di Sipio C., Di Ruscio D., Di Penta M. (2021) Recommending api function calls and code snippets to support software development. IEEE Trans Softw Eng, 1\u20131. https:\/\/doi.org\/10.1109\/TSE.2021.3059907","DOI":"10.1109\/TSE.2021.3059907"},{"key":"9963_CR45","doi-asserted-by":"crossref","unstructured":"Nguyen P.T., Di Rocco J., Rubei R., Di Ruscio D. (2018) Crosssim: exploiting mutual relationships to detect similar OSS projects. In: 2018 44th euromicro conference on software engineering and advanced applications (SEAA), pp 388\u2013395","DOI":"10.1109\/SEAA.2018.00069"},{"key":"9963_CR46","doi-asserted-by":"publisher","unstructured":"Nguyen P.T., Di Rocco J., Rubei R., Di Ruscio D. (2020) An automated approach to assess the similarity of GitHub repositories. Software Quality Journal. https:\/\/doi.org\/10.1007\/s11219-019-09483-0","DOI":"10.1007\/s11219-019-09483-0"},{"key":"9963_CR47","doi-asserted-by":"crossref","unstructured":"Nguyen P.T., Tomeo P., Di Noia T., Di Sciascio E. (2015) Content-based recommendations via DBpedia and freebase: a case study in the music domain. In: Proceedings of the 14th international conference on the semantic web - ISWC 2015 - volume 9366. Springer, New York, pp 605\u2013621","DOI":"10.1007\/978-3-319-25007-6_35"},{"key":"9963_CR48","doi-asserted-by":"publisher","first-page":"127","DOI":"10.1016\/j.jss.2016.07.026","volume":"129","author":"H Niu","year":"2017","unstructured":"Niu H., Keivanloo I., Zou Y. (2017) API usage pattern recommendation for software development. Journal of Systems Software 129:127\u2013139","journal-title":"Journal of Systems Software"},{"key":"9963_CR49","doi-asserted-by":"publisher","unstructured":"Noia T.D., Ostuni V.C. (2015) Recommender systems and linked open data. In: Faber W., Paschke A. (eds) Reasoning Web. Web Logic Rules - 11th International Summer School 2015, Berlin, Germany, July 31 - August 4, 2015, Tutorial Lectures, Lecture Notes in Computer Science. https:\/\/doi.org\/10.1007\/978-3-319-21768-0_4, vol 9203. Springer, pp 88\u2013113","DOI":"10.1007\/978-3-319-21768-0_4"},{"issue":"C","key":"9963_CR50","doi-asserted-by":"publisher","first-page":"55","DOI":"10.1016\/j.infsof.2016.11.007","volume":"83","author":"A Ouni","year":"2017","unstructured":"Ouni A., Kula R.G., Kessentini M., Ishio T., German D.M., Inoue K. (2017) Search-based software library recommendation using multi-objective optimization. Information and Software Technology 83(C):55\u201375","journal-title":"Information and Software Technology"},{"key":"9963_CR51","doi-asserted-by":"crossref","unstructured":"Pakdeetrakulwong U., Wongthongtham P., Siricharoen W.V. (2014) Recommendation systems for software engineering: a survey from software development life cycle phase perspective. pp. 137\u2013142. IEEE","DOI":"10.1109\/ICITST.2014.7038793"},{"issue":"3","key":"9963_CR52","doi-asserted-by":"publisher","first-page":"331","DOI":"10.1108\/13522750810879048","volume":"11","author":"S Pettigrew","year":"2008","unstructured":"Pettigrew S., Charters S. (2008) Tasting as a projective technique. Qualitative Market Research: An International Journal 11(3):331\u2013343","journal-title":"Qualitative Market Research: An International Journal"},{"key":"9963_CR53","doi-asserted-by":"publisher","unstructured":"Ponzanelli L., Bavota G., Di Penta M., Oliveto R., Lanza M. (2016) Prompter: turning the IDE into a self-confident programming assistant. Empirical Software Engineering 21(5), 2190\u20132231. https:\/\/doi.org\/10.1007\/s10664-015-9397-1. http:\/\/link.springer.com\/10.1007\/s10664-015-9397-1","DOI":"10.1007\/s10664-015-9397-1"},{"issue":"5","key":"9963_CR54","doi-asserted-by":"publisher","first-page":"2190","DOI":"10.1007\/s10664-015-9397-1","volume":"21","author":"L Ponzanelli","year":"2016","unstructured":"Ponzanelli L., Bavota G., Penta M.D., Oliveto R., Lanza M. (2016) Prompter - turning the IDE into a self-confident programming assistant. Empir Softw Eng 21(5):2190\u20132231","journal-title":"Empir Softw Eng"},{"key":"9963_CR55","doi-asserted-by":"publisher","unstructured":"Proksch S., Bauer V., Murphy G.C. (2014) How to build a recommendation system for software engineering. In: Meyer B., Nordio M. (eds) Software engineering - international summer schools, LASER 2013-2014, Elba, Italy, Revised Tutorial Lectures, Lecture Notes in Computer Science. https:\/\/doi.org\/10.1007\/978-3-319-28406-4_1, vol 8987. Springer, pp 1\u201342","DOI":"10.1007\/978-3-319-28406-4_1"},{"key":"9963_CR56","doi-asserted-by":"crossref","unstructured":"Proksch S., Bauer V., Murphy G.C. (2015) How to build a recommendation system for software engineering. In: Meyer B., Nordio M. (eds) Advances in the theory and practice of software engineering - LASER 2013-2014, LNCS. http:\/\/tubiblio.ulb.tu-darmstadt.de\/77729\/, vol 8987. Springer, pp 1\u201342","DOI":"10.1007\/978-3-319-28406-4_1"},{"issue":"5","key":"9963_CR57","doi-asserted-by":"publisher","first-page":"613","DOI":"10.1109\/TSE.2012.63","volume":"39","author":"MP Robillard","year":"2013","unstructured":"Robillard M.P., Bodden E., Kawrykow D., Mezini M., Ratchford T. (2013) Automated API property inference techniques. IEEE Transactions on Software Engineering 39(5):613\u2013637","journal-title":"IEEE Transactions on Software Engineering"},{"key":"9963_CR58","doi-asserted-by":"crossref","unstructured":"Robillard M.P., Maalej W., Walker R.J., Zimmermann T. (eds) (2014) Recommendation systems in software engineering. Springer, Berlin","DOI":"10.1007\/978-3-642-45135-5"},{"key":"9963_CR59","doi-asserted-by":"publisher","unstructured":"Rubei R., Di Sipio C., Nguyen P.T., Di Rocco J., Di Ruscio D. (2020) PostFinder: mining stack overflow posts to support software developers. Information and Software Technology 127, 106367. https:\/\/doi.org\/10.1016\/j.infsof.2020.106367. http:\/\/www.sciencedirect.com\/science\/article\/pii\/S0950584920301361","DOI":"10.1016\/j.infsof.2020.106367"},{"key":"9963_CR60","doi-asserted-by":"publisher","first-page":"164","DOI":"10.1016\/j.jss.2018.08.032","volume":"145","author":"MA Saied","year":"2018","unstructured":"Saied M.A., Ouni A., Sahraoui H., Kula R.G., Inoue K., Lo D. (2018) Improving reusability of software libraries through usage pattern mining. Journal of Systems Software 145:164\u2013179","journal-title":"Journal of Systems Software"},{"key":"9963_CR61","doi-asserted-by":"publisher","unstructured":"Schedl M., Zamani H., Chen C., Deldjoo Y., Elahi M. (2018) Current challenges and visions in music recommender systems research. International Journal Multimedia Information Retrieval 7(2), 95\u2013116. https:\/\/doi.org\/10.1007\/s13735-018-0154-2","DOI":"10.1007\/s13735-018-0154-2"},{"issue":"1","key":"9963_CR62","doi-asserted-by":"publisher","first-page":"72","DOI":"10.2307\/1412159","volume":"15","author":"C Spearman","year":"1904","unstructured":"Spearman C. (1904) The proof and measurement of association between two things. The American Journal of Psychology 15(1):72\u2013101","journal-title":"The American Journal of Psychology"},{"issue":"1","key":"9963_CR63","doi-asserted-by":"publisher","first-page":"28","DOI":"10.1109\/MS.2004.1259204","volume":"21","author":"D Spinellis","year":"2004","unstructured":"Spinellis D., Szyperski C. (2004) How is open source affecting software development? IEEE Software 21(1):28\u201333","journal-title":"IEEE Software"},{"key":"9963_CR64","doi-asserted-by":"crossref","unstructured":"Thung F., Lo D., Lawall J. (2013) Automated library recommendation. In: 2013 20th working conf. on reverse engineering (WCRE), pp 182\u2013191","DOI":"10.1109\/WCRE.2013.6671293"},{"key":"9963_CR65","doi-asserted-by":"crossref","unstructured":"Vargas S., Castells P. (2011) Rank and relevance in novelty and diversity metrics for recommender systems. In: Proceedings of the fifth ACM conference on recommender systems, RecSys \u201911. ACM, New York, pp 109\u2013116","DOI":"10.1145\/2043932.2043955"},{"key":"9963_CR66","doi-asserted-by":"crossref","unstructured":"Vargas S., Castells P. (2014) Improving sales diversity by recommending users to items. In: Eighth ACM conference on recommender systems, recsys \u201914, Foster City, Silicon Valley, CA, USA - October 06 - 10, 2014, pp 145\u2013152","DOI":"10.1145\/2645710.2645744"},{"key":"9963_CR67","doi-asserted-by":"publisher","unstructured":"Wang J., Dang Y., Zhang H., Chen K., Xie T., Zhang D. (2013) Mining succinct and high-coverage API usage patterns from source code. In: 10th working conference on mining software repositories. https:\/\/doi.org\/10.1109\/MSR.2013.6624045. IEEE, Piscataway, pp 319\u2013328","DOI":"10.1109\/MSR.2013.6624045"},{"issue":"1","key":"9963_CR68","doi-asserted-by":"publisher","first-page":"67","DOI":"10.1109\/4235.585893","volume":"1","author":"D Wolpert","year":"1997","unstructured":"Wolpert D., Macready W. (1997) No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation 1(1):67\u201382. https:\/\/doi.org\/10.1109\/4235.585893. http:\/\/ieeexplore.ieee.org\/document\/585893\/","journal-title":"IEEE Transactions on Evolutionary Computation"},{"issue":"9","key":"9963_CR69","doi-asserted-by":"publisher","first-page":"2839","DOI":"10.1016\/j.patcog.2015.03.009","volume":"48","author":"TT Wong","year":"2015","unstructured":"Wong T.T. (2015) Performance evaluation of classification algorithms by K-fold and leave-one-out cross validation. Pattern Recognition 48(9):2839\u20132846. https:\/\/doi.org\/10.1016\/j.patcog.2015.03.009","journal-title":"Pattern Recognition"},{"key":"9963_CR70","unstructured":"Wu L., Shah S., Choi S., Tiwari M., Posse C. (2014) The browsemaps: Collaborative filtering at LinkedIn. In: RSWEb@recsys, CEUR workshop proceedings. CEUR-WS.org, vol 1271"},{"key":"9963_CR71","doi-asserted-by":"crossref","unstructured":"Zhang Y., Lo D., Kochhar P.S., Xia X., Li Q., Sun J. (2017) Detecting similar repositories on GitHub. In: 2017 IEEE 24th international conference on software analysis, evolution and reengineering (SANER), vol 00, pp 13\u201323","DOI":"10.1109\/SANER.2017.7884605"},{"key":"9963_CR72","unstructured":"Zheng M., Pan X., Lillis D. (2018) CodEX: source code plagiarism detection based on abstract syntax tree. In: Proceedings for the 26th AIAI irish conference on artificial intelligence and cognitive science trinity college Dublin, Dublin, Ireland, December 6-7th, 2018, pp 362\u2013373"},{"key":"9963_CR73","doi-asserted-by":"crossref","unstructured":"Zhong H., Xie T., Zhang L., Pei J., Mei H. (2009) MAPO: mining and recommending API usage patterns. In: 23rd European conference on object-oriented programming. Springer, Berlin, pp 318\u2013343","DOI":"10.1007\/978-3-642-03013-0_15"}],"container-title":["Empirical Software Engineering"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-021-09963-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/article\/10.1007\/s10664-021-09963-7\/fulltext.html","content-type":"text\/html","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/link.springer.com\/content\/pdf\/10.1007\/s10664-021-09963-7.pdf","content-type":"application\/pdf","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2021,6,23]],"date-time":"2021-06-23T12:31:18Z","timestamp":1624451478000},"score":1,"resource":{"primary":{"URL":"https:\/\/link.springer.com\/10.1007\/s10664-021-09963-7"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2021,5,14]]},"references-count":73,"journal-issue":{"issue":"4","published-print":{"date-parts":[[2021,7]]}},"alternative-id":["9963"],"URL":"https:\/\/doi.org\/10.1007\/s10664-021-09963-7","relation":{},"ISSN":["1382-3256","1573-7616"],"issn-type":[{"value":"1382-3256","type":"print"},{"value":"1573-7616","type":"electronic"}],"subject":[],"published":{"date-parts":[[2021,5,14]]},"assertion":[{"value":"19 March 2021","order":1,"name":"accepted","label":"Accepted","group":{"name":"ArticleHistory","label":"Article History"}},{"value":"14 May 2021","order":2,"name":"first_online","label":"First Online","group":{"name":"ArticleHistory","label":"Article History"}}],"article-number":"69"}}