{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,26]],"date-time":"2026-03-26T21:01:26Z","timestamp":1774558886521,"version":"3.50.1"},"reference-count":28,"publisher":"Emerald","issue":"5","license":[{"start":{"date-parts":[[2008,11,21]],"date-time":"2008-11-21T00:00:00Z","timestamp":1227225600000},"content-version":"tdm","delay-in-days":0,"URL":"https:\/\/www.emerald.com\/insight\/site-policies"}],"content-domain":{"domain":[],"crossmark-restriction":false},"short-container-title":[],"published-print":{"date-parts":[[2008,11,21]]},"abstract":"<jats:sec><jats:title content-type=\"abstract-heading\">Purpose<\/jats:title><jats:p>The purpose of this paper is to develop and test a model of the relative performance of open source software (OSS) projects.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Design\/methodology\/approach<\/jats:title><jats:p>This paper evaluates the relative performance of OSS projects by evaluating multiple project inputs and multiple project outputs by using a data envelopment analysis (DEA) model. The DEA model produces an efficiency score for each project based on project inputs and outputs. The method of producing an efficiency score is based on the convex envelopment technology structure. The efficiency measure quantifies a \u201cdistance\u201d to an efficient frontier.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Findings<\/jats:title><jats:p>The DEA model produced an index of corresponding intensities linking an inefficient project to its benchmark efficient project(s). The inefficiency measures produced an ordering of inefficient projects. Eight projects were found to be \u201cefficient\u201d and used as benchmarking projects.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Research limitations\/implications<\/jats:title><jats:p>This research is limited to only security\u2010based OSS projects. Future research on other areas of OSS projects is warranted.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Practical implications<\/jats:title><jats:p>The result of this research is a practical model that can be used by OSS project developers to evaluate the relative performance of their projects and make resource decisions.<\/jats:p><\/jats:sec><jats:sec><jats:title content-type=\"abstract-heading\">Originality\/value<\/jats:title><jats:p>This research extends the work of previous studies that have examined the relative performance of software development projects in a traditional development environment. As a result of this research, OSS projects can now be adequately benchmarked and evaluated according to project performance. An OSS project manger can effectively use these results to critically evaluate resources for their project and judge the relative efficiency of the resources.<\/jats:p><\/jats:sec>","DOI":"10.1108\/09685220810920530","type":"journal-article","created":{"date-parts":[[2008,11,15]],"date-time":"2008-11-15T07:03:03Z","timestamp":1226732583000},"page":"449-462","source":"Crossref","is-referenced-by-count":17,"title":["Evaluating the performance of open source software projects using data envelopment analysis"],"prefix":"10.1108","volume":"16","author":[{"given":"Barry","family":"Wray","sequence":"first","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Richard","family":"Mathieu","sequence":"additional","affiliation":[],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"140","reference":[{"key":"key2022020719550057900_b2","doi-asserted-by":"crossref","unstructured":"Banker, R.D. and Kemerer, C.F. (1989), \u201cScale economies in new software development\u201d, IEEE Transactions on Software Development, Vol. 15 No. 10, pp. 1199\u2010205.","DOI":"10.1109\/TSE.1989.559768"},{"key":"key2022020719550057900_b1","doi-asserted-by":"crossref","unstructured":"Banker, R.D., Datar, S.M. and Kemerer, C.F. (1991), \u201cA model to evaluate variables impacting the production of software maintenance projects\u201d, Management Science, Vol. 37 No. 1, pp. 1\u20108.","DOI":"10.1287\/mnsc.37.1.1"},{"key":"key2022020719550057900_b4","doi-asserted-by":"crossref","unstructured":"Charnes, A., Rousseau, J. and Semple, J. (1996), \u201cSensitivity and stability of efficiency classifications in data envelopment analysis\u201d, Journal of Productivity Analysis, Vol. 7 No. 1, pp. 5\u201018.","DOI":"10.1007\/BF00158473"},{"key":"key2022020719550057900_b3","doi-asserted-by":"crossref","unstructured":"Charnes, A., Cooper, W.W., Golany, B., Seiford, L. and Stutz, J. (1985), \u201cFoundations of data envelopment analysis for Pareto\u2010Koopmans efficient empirical production functions\u201d, Journal of Econometrics, Vol. 30, pp. 91\u2010107.","DOI":"10.1016\/0304-4076(85)90133-2"},{"key":"key2022020719550057900_b5","doi-asserted-by":"crossref","unstructured":"Colbert, A., Levary, R. and Shaner, M. (2000), \u201cDetermining the relative efficiency of MBA programs using DEA\u201d, European Journal of Operational Research, Vol. 125, pp. 656\u201069.","DOI":"10.1016\/S0377-2217(99)00275-1"},{"key":"key2022020719550057900_b6","unstructured":"Crowston, K., Annabi, H. and Howison, J. (2003), \u201cDefining open source software project success\u201d, paper presented at 24th International Conference on Information Systems (ICIS), pp. 327\u201040."},{"key":"key2022020719550057900_b7","unstructured":"Elam, J. (1991), \u201cEvaluating the efficiency of IS organizations using data envelope analysis\u201d, Proceedings of the International Function Point Users Group 1991 Fall Conference, October."},{"key":"key2022020719550057900_b8","doi-asserted-by":"crossref","unstructured":"Farrell, M. (1957), \u201cThe measurement of productive efficiency\u201d, Journal of the Royal Statistical Society, Series A, Vol. 120 No. 3, pp. 253\u201090.","DOI":"10.2307\/2343100"},{"key":"key2022020719550057900_b9","unstructured":"Feller, J. and Fitzgerald, B. (2002), Understanding Open Source Software Development, Addison Wesley, London."},{"key":"key2022020719550057900_b10","doi-asserted-by":"crossref","unstructured":"Flitman, A. (2003), \u201cTowards meaningful benchmarking of software development team productivity\u201d, Benchmarking: An International Journal, Vol. 10 No. 4, pp. 382\u201099.","DOI":"10.1108\/146357703104484999"},{"key":"key2022020719550057900_b28","unstructured":"Greenemeier, L. (2005), \u201cOPEN\u2010SOURCE EXUBERANCE \u2013 hot\u2010shot entrepreneurs, VC money, even paradigm shifts. Dot\u2010coms revisited? No, it's today's flood of open\u2010source startups\u201d, Information Week, July 11."},{"key":"key2022020719550057900_b11","unstructured":"Hahn, J. and Zhang, C. (2005), \u201cAn exploratory study of open source projects from a project management perspective\u201d, paper presented at MIS Research Workshop, Purdue University, West Lafayette, IN."},{"key":"key2022020719550057900_b12","doi-asserted-by":"crossref","unstructured":"Jorgensen, N. (2001), \u201cPutting it all in the trunk: incremental software development in the FreeBSD open source project\u201d, Information Systems Journal, Vol. 11 No. 4, pp. 321\u201036.","DOI":"10.1046\/j.1365-2575.2001.00113.x"},{"key":"key2022020719550057900_b13","unstructured":"Kalina, I. and Czyzycki, A. (2005), \u201cThe ins and outs of open source\u201d, Consulting to Management, Vol. 16 No. 3, pp. 41\u20107."},{"key":"key2022020719550057900_b14","doi-asserted-by":"crossref","unstructured":"Kogut, B. and Metiu, A. (2001), \u201cOpen source software development and distributed innovation\u201d, Oxford Review of Economic Policy, Vol. 17 No. 2, pp. 248\u201064.","DOI":"10.1093\/oxrep\/17.2.248"},{"key":"key2022020719550057900_b15","unstructured":"Lacy, S. (2005), \u201cOpen source: now it's an ecosystem\u201d, BusinessWeek Online, October, p. 3."},{"key":"key2022020719550057900_b16","unstructured":"Mogull, R. and Girard, J. (2006), \u201cAssessment methodology for open\u2010source security testing tools\u201d, Gartner Research, (ID G00144478), December."},{"key":"key2022020719550057900_b17","unstructured":"Mookhey, K.K. (2004), \u201cOpen source tools for security and control assessment\u201d, Information Systems Control Journal, Vol. 1, pp. 39\u201044."},{"key":"key2022020719550057900_b18","doi-asserted-by":"crossref","unstructured":"Paradi, J.C., Reese, D.N. and Rosen, D. (1997), \u201cApplications of DEA to measure the efficiency of software production at two large Canadian banks\u201d, Annals of Operations Research, Vol. 73, pp. 91\u2010115.","DOI":"10.1023\/A:1018953900977"},{"key":"key2022020719550057900_b19","doi-asserted-by":"crossref","unstructured":"Payne, C. (2002), \u201cOn the security of open source software\u201d, Information Systems Journal, Vol. 12 No. 1, pp. 61\u201078.","DOI":"10.1046\/j.1365-2575.2002.00118.x"},{"key":"key2022020719550057900_b20","unstructured":"Salkever, A. (2001), \u201cIs open\u2010source security software safe?\u201d, BusinessWeek Online, December, p. 12."},{"key":"key2022020719550057900_b21","unstructured":"Scacchi, W. (2002), \u201cIs open source software development faster, better and cheaper than software engineering\u201d, paper presented at 2nd ICSE Workshop on Open Source Software Engineering, Orlando, FL, May."},{"key":"key2022020719550057900_b22","doi-asserted-by":"crossref","unstructured":"Scacchi, W. (2004), \u201cFree and open source development practices in the game community\u201d, IEEE Software, Vol. 21 No. 1, pp. 59\u201066.","DOI":"10.1109\/MS.2004.1259221"},{"key":"key2022020719550057900_b23","doi-asserted-by":"crossref","unstructured":"Scacchi, W., Feller, J., Fitzgerald, B., Hissam, S. and Lakhani, K. (2006), \u201cUnderstanding free\/open source software development processes\u201d, Software Process Improvement and Practice, Vol. 11, pp. 95\u2010105.","DOI":"10.1002\/spip.255"},{"key":"key2022020719550057900_b24","unstructured":"Scheel, H. (2000), EMS: Efficiency Measurement System User's Manual, available at: www.wiso.uni\u2010dortmund.de\/lsfg\/or\/scheel\/ems\/."},{"key":"key2022020719550057900_b25","doi-asserted-by":"crossref","unstructured":"Stamelos, I., Angelis, L., Oikonomou, A. and Bleris, G. (2002), \u201cCode quality analysis in open source software development\u201d, Information Systems Journal, Vol. 12, pp. 43\u201060.","DOI":"10.1046\/j.1365-2575.2002.00117.x"},{"key":"key2022020719550057900_b26","doi-asserted-by":"crossref","unstructured":"Stensrud, E. and Myrtveit, I. (2003), \u201cIdentifying high performance ERP projects\u201d, IEEE Transactions on Software Engineering, Vol. 29 No. 5, pp. 398\u2010415.","DOI":"10.1109\/TSE.2003.1199070"},{"key":"key2022020719550057900_b27","doi-asserted-by":"crossref","unstructured":"Yang, Z. and Paradi, J.C. (2004), \u201cDEA evaluation of Y2K software retrofit program\u201d, IEEE Transactions on Engineering Management, Vol. 51 No. 3, pp. 279\u201087.","DOI":"10.1109\/TEM.2004.830843"}],"container-title":["Information Management &amp; Computer Security"],"original-title":[],"language":"en","link":[{"URL":"http:\/\/www.emeraldinsight.com\/doi\/full-xml\/10.1108\/09685220810920530","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/09685220810920530\/full\/xml","content-type":"application\/xml","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/www.emerald.com\/insight\/content\/doi\/10.1108\/09685220810920530\/full\/html","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,7,25]],"date-time":"2025-07-25T00:09:07Z","timestamp":1753402147000},"score":1,"resource":{"primary":{"URL":"http:\/\/www.emerald.com\/ics\/article\/16\/5\/449-462\/185647"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2008,11,21]]},"references-count":28,"journal-issue":{"issue":"5","published-print":{"date-parts":[[2008,11,21]]}},"alternative-id":["10.1108\/09685220810920530"],"URL":"https:\/\/doi.org\/10.1108\/09685220810920530","relation":{},"ISSN":["0968-5227"],"issn-type":[{"value":"0968-5227","type":"print"}],"subject":[],"published":{"date-parts":[[2008,11,21]]}}}