{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2026,3,11]],"date-time":"2026-03-11T01:25:01Z","timestamp":1773192301273,"version":"3.50.1"},"reference-count":54,"publisher":"Association for Computing Machinery (ACM)","issue":"1","license":[{"start":{"date-parts":[[2011,12,1]],"date-time":"2011-12-01T00:00:00Z","timestamp":1322697600000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["ACM Trans. Softw. Eng. Methodol."],"published-print":{"date-parts":[[2011,12]]},"abstract":"<jats:p>Software developers search the Web for various kinds of source code for diverse reasons. In a previous study, we found that searches varied along two dimensions: the size of the search target (e.g., block, subsystem, or system) and the motivation for the search (e.g., reference example or as-is reuse). Would each of these kinds of searches require different search technologies? To answer this question, we conducted an experiment with 36 participants to evaluate three diverse approaches (general purpose information retrieval, source code search, and component reuse), as represented by five Web sites (Google, Koders, Krugle, Google Code Search, and SourceForge). The independent variables were search engine, size of search target, and motivation for search. The dependent variable was the participants judgement of the relevance of the first ten hits. We found that it was easier to find reference examples than components for as-is reuse and that participants obtained the best results using a general-purpose information retrieval site. However, we also found an interaction effect: code-specific search engines worked better in searches for subsystems, but Google worked better on searches for blocks. These results can be used to guide the creation of new tools for retrieving source code from the Web.<\/jats:p>","DOI":"10.1145\/2063239.2063243","type":"journal-article","created":{"date-parts":[[2011,12,20]],"date-time":"2011-12-20T17:49:14Z","timestamp":1324403354000},"page":"1-25","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":74,"title":["How Well Do Search Engines Support Code Retrieval on the Web?"],"prefix":"10.1145","volume":"21","author":[{"given":"Susan Elliott","family":"Sim","sequence":"first","affiliation":[{"name":"University of California, Irvine"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Medha","family":"Umarji","sequence":"additional","affiliation":[{"name":"University of Maryland, Baltimore County"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Sukanya","family":"Ratanotayanon","sequence":"additional","affiliation":[{"name":"University of California, Irvine"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Cristina V.","family":"Lopes","sequence":"additional","affiliation":[{"name":"University of California, Irvine"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2011,12]]},"reference":[{"key":"e_1_2_2_1_1","volume-title":"Proceedings of the Conference on Functional Programming Languages and Computer Architecture. Springer, 427--447","author":"Aiken A.","unstructured":"Aiken , A. and Murphy , B. R . 1991. Implementing regular tree expressions . In Proceedings of the Conference on Functional Programming Languages and Computer Architecture. Springer, 427--447 . Aiken, A. and Murphy, B. R. 1991. Implementing regular tree expressions. In Proceedings of the Conference on Functional Programming Languages and Computer Architecture. Springer, 427--447."},{"key":"e_1_2_2_2_1","doi-asserted-by":"publisher","DOI":"10.5555\/832306.837053"},{"key":"e_1_2_2_3_1","doi-asserted-by":"publisher","DOI":"10.1145\/1134285.1134288"},{"key":"e_1_2_2_4_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(199607)47:7%3C493::AID-ASI3%3E3.3.CO;2-Y"},{"key":"e_1_2_2_5_1","doi-asserted-by":"publisher","DOI":"10.1109\/MC.1987.1663532"},{"key":"e_1_2_2_6_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.48940"},{"key":"e_1_2_2_7_1","volume-title":"Statistical Power Analysis for the Behavioral Sciences","author":"Cohen J.","unstructured":"Cohen , J. 1988. Statistical Power Analysis for the Behavioral Sciences 2 nd Ed. Lawrence Erlbaum Associates , Philadelphia, PA . Cohen, J. 1988. Statistical Power Analysis for the Behavioral Sciences 2nd Ed. Lawrence Erlbaum Associates, Philadelphia, PA.","edition":"2"},{"key":"e_1_2_2_8_1","volume-title":"Proceedings of the 13th Text REtrieval Conference. NIST","author":"Craswell N.","unstructured":"Craswell , N. and Hawking , D . 2004. Overview of the trec 2004 Webl track . In Proceedings of the 13th Text REtrieval Conference. NIST , Gaithersburg, MD, 1--9. Craswell, N. and Hawking, D. 2004. Overview of the trec 2004 Webl track. In Proceedings of the 13th Text REtrieval Conference. NIST, Gaithersburg, MD, 1--9."},{"key":"e_1_2_2_9_1","volume-title":"Proceedings of the 4th IFIP International Conference on Open Source Systems (OSS\u201908)","author":"Deshpande A.","unstructured":"Deshpande , A. and Riehle , D . 2008. The total growth of open source . In Proceedings of the 4th IFIP International Conference on Open Source Systems (OSS\u201908) . Springer. Deshpande, A. and Riehle, D. 2008. The total growth of open source. In Proceedings of the 4th IFIP International Conference on Open Source Systems (OSS\u201908). Springer."},{"key":"e_1_2_2_10_1","doi-asserted-by":"publisher","DOI":"10.3758\/BF03193146"},{"key":"e_1_2_2_11_1","volume-title":"Proceedings of the 13th International Conference on Software Engineering. IEEE Computer Society Press","author":"Fischer G.","unstructured":"Fischer , G. , Henninger , S. , and Redmiles , D . 1991. Cognitive tools for locating and comprehending software objects for reuse . In Proceedings of the 13th International Conference on Software Engineering. IEEE Computer Society Press , Los Alamitos, CA, 318--328. Fischer, G., Henninger, S., and Redmiles, D. 1991. Cognitive tools for locating and comprehending software objects for reuse. In Proceedings of the 13th International Conference on Software Engineering. IEEE Computer Society Press, Los Alamitos, CA, 318--328."},{"key":"e_1_2_2_12_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2005.85"},{"key":"e_1_2_2_13_1","doi-asserted-by":"publisher","DOI":"10.1145\/1094811.1094819"},{"key":"e_1_2_2_14_1","doi-asserted-by":"publisher","DOI":"10.1109\/MSR.2007.9"},{"key":"e_1_2_2_15_1","volume-title":"CSTR 2006-14","author":"Hartmann B.","unstructured":"Hartmann , B. , Doorley , S. , and Klemmer , S. R . 2006. Hacking, mashing, gluing: A study of opportunistic design. Tech. rep . CSTR 2006-14 , Department of Computer Science, Stanford University. Hartmann, B., Doorley, S., and Klemmer, S. R. 2006. Hacking, mashing, gluing: A study of opportunistic design. Tech. rep. CSTR 2006-14, Department of Computer Science, Stanford University."},{"key":"e_1_2_2_16_1","doi-asserted-by":"publisher","DOI":"10.1145\/1370750.1370771"},{"key":"e_1_2_2_17_1","doi-asserted-by":"publisher","DOI":"10.1145\/1294211.1294216"},{"key":"e_1_2_2_18_1","doi-asserted-by":"publisher","DOI":"10.1145\/1062455.1062491"},{"key":"e_1_2_2_19_1","doi-asserted-by":"publisher","DOI":"10.1109\/ICSE.2007.83"},{"key":"e_1_2_2_20_1","doi-asserted-by":"publisher","DOI":"10.1007\/978-3-540-68073-4_35"},{"key":"e_1_2_2_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/1081706.1081744"},{"key":"e_1_2_2_22_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2008.110"},{"key":"e_1_2_2_23_1","doi-asserted-by":"crossref","unstructured":"Kamp P.-H. 2004. Keep in touch! IEEE Softw. 21 1 46--47. Kamp P.-H. 2004. Keep in touch! IEEE Softw. 21 1 46--47.","DOI":"10.1109\/MS.2004.1320860"},{"key":"e_1_2_2_24_1","doi-asserted-by":"publisher","DOI":"10.1016\/j.infsof.2007.02.015"},{"key":"e_1_2_2_25_1","doi-asserted-by":"crossref","unstructured":"Langville A. and Meyer C. 2006. Google\u2019s Pagerank and Beyond: The Science of Search Engine Rankings. Princeton University Press Princeton NJ. Langville A. and Meyer C. 2006. Google\u2019s Pagerank and Beyond: The Science of Search Engine Rankings . Princeton University Press Princeton NJ.","DOI":"10.1515\/9781400830329"},{"key":"e_1_2_2_26_1","doi-asserted-by":"publisher","DOI":"10.1145\/1297846.1297944"},{"key":"e_1_2_2_27_1","doi-asserted-by":"publisher","DOI":"10.1145\/390010.808258"},{"key":"e_1_2_2_28_1","doi-asserted-by":"publisher","DOI":"10.1145\/1065010.1065018"},{"key":"e_1_2_2_29_1","doi-asserted-by":"crossref","unstructured":"Manning C. D. Raghavan P. and Sch\u00fctze H. 2008. Introduction to Information Retrieval. Cambridge University Press. Manning C. D. Raghavan P. and Sch\u00fctze H. 2008. Introduction to Information Retrieval . Cambridge University Press.","DOI":"10.1017\/CBO9780511809071"},{"key":"e_1_2_2_30_1","volume-title":"Proceedings of the 11th Working Conference on Reverse Engineering. IEEE Computer Society Press","author":"Marcus A.","unstructured":"Marcus , A. , Sergeyev , A. , Rajlich , V. , and Maletic , J . 2004. An information retrieval approach to concept location in source code . In Proceedings of the 11th Working Conference on Reverse Engineering. IEEE Computer Society Press , Los Alamitos, CA, 214--223. Marcus, A., Sergeyev, A., Rajlich, V., and Maletic, J. 2004. An information retrieval approach to concept location in source code. In Proceedings of the 11th Working Conference on Reverse Engineering. IEEE Computer Society Press, Los Alamitos, CA, 214--223."},{"key":"e_1_2_2_31_1","doi-asserted-by":"publisher","DOI":"10.5555\/590631.590637"},{"key":"e_1_2_2_32_1","doi-asserted-by":"publisher","DOI":"10.1145\/234426.234441"},{"key":"e_1_2_2_33_1","volume-title":"Proceedings of the Onward Track at OOPSLA.","volume":"2","author":"Noble J.","unstructured":"Noble , J. and Biddle , R . 2002. Notes on postmodern programming . In Proceedings of the Onward Track at OOPSLA. vol. 2 . ACM Press, New York, NY, 49--71. Noble, J. and Biddle, R. 2002. Notes on postmodern programming. In Proceedings of the Onward Track at OOPSLA. vol. 2. ACM Press, New York, NY, 49--71."},{"key":"e_1_2_2_34_1","unstructured":"Northrop L. Feiler P. Gabriel R. P. Goodenough J. Linger R. Longstaff T. Kazman R. Klein M. Schmidt D. Sullivan K. and Wallnau K. 2006. Ultra-large-scale systems: The software challenge of the future. Tech. rep. Software Engineering Institute Carnegie Mellon University. Northrop L. Feiler P. Gabriel R. P. Goodenough J. Linger R. Longstaff T. Kazman R. Klein M. Schmidt D. Sullivan K. and Wallnau K. 2006. Ultra-large-scale systems: The software challenge of the future. Tech. rep. Software Engineering Institute Carnegie Mellon University."},{"key":"e_1_2_2_35_1","doi-asserted-by":"publisher","DOI":"10.1109\/2.910904"},{"key":"e_1_2_2_36_1","doi-asserted-by":"publisher","DOI":"10.1109\/32.295894"},{"key":"e_1_2_2_37_1","doi-asserted-by":"publisher","DOI":"10.1145\/103167.103176"},{"key":"e_1_2_2_38_1","doi-asserted-by":"publisher","DOI":"10.1145\/859670.859678"},{"key":"e_1_2_2_39_1","volume-title":"The Volunteer Subject","author":"Rosenthal R.","unstructured":"Rosenthal , R. 1983. The Volunteer Subject . Ardent Media Inc . Rosenthal, R. 1983. The Volunteer Subject. Ardent Media Inc."},{"key":"e_1_2_2_40_1","doi-asserted-by":"publisher","DOI":"10.1145\/1167473.1167508"},{"key":"e_1_2_2_41_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.v58:13"},{"key":"e_1_2_2_42_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.v58:13"},{"key":"e_1_2_2_43_1","volume-title":"Statisical Reasoning for the Behavioral Sciences","author":"Shavelson R. J.","unstructured":"Shavelson , R. J. 1996. Statisical Reasoning for the Behavioral Sciences 3 rd Ed. Allyn and Bacon . Shavelson, R. J. 1996. Statisical Reasoning for the Behavioral Sciences 3rd Ed. Allyn and Bacon.","edition":"3"},{"key":"e_1_2_2_44_1","volume-title":"Proceedings of the 6th International Workshop on Program Comprehension. IEEE Computer Society","author":"Sim S. E.","unstructured":"Sim , S. E. , Clarke , C. L. A. , and Holt , R. C . 1998. Archetypal source code searches: A survey of software developers and maintainers . In Proceedings of the 6th International Workshop on Program Comprehension. IEEE Computer Society , Los Alamitos, CA, 180. Sim, S. E., Clarke, C. L. A., and Holt, R. C. 1998. Archetypal source code searches: A survey of software developers and maintainers. In Proceedings of the 6th International Workshop on Program Comprehension. IEEE Computer Society, Los Alamitos, CA, 180."},{"key":"e_1_2_2_45_1","unstructured":"Singer J. and Lethbridge T. 1997. What\u2019s so great about \u2018grep\u2019? implications for program comprehension tools. Tech. rep. National Research Council Canada. Singer J. and Lethbridge T. 1997. What\u2019s so great about \u2018grep\u2019? implications for program comprehension tools. Tech. rep. National Research Council Canada."},{"key":"e_1_2_2_46_1","doi-asserted-by":"publisher","DOI":"10.1109\/MS.2004.1259204"},{"key":"e_1_2_2_47_1","doi-asserted-by":"publisher","DOI":"10.1145\/363347.363387"},{"key":"e_1_2_2_48_1","doi-asserted-by":"publisher","DOI":"10.1145\/1321631.1321663"},{"key":"e_1_2_2_49_1","volume-title":"Proceedings of the Working Group on Open Source Software (OSS) at the IFIP 20th World Computer Congress. B. Russo, Ed","author":"Umarji M.","unstructured":"Umarji , M. , Sim , S. E. , and Lopes , C. V . 2008. Archetypal internet-scale source code searching . In Proceedings of the Working Group on Open Source Software (OSS) at the IFIP 20th World Computer Congress. B. Russo, Ed . Springer, New York, NY, 7. Umarji, M., Sim, S. E., and Lopes, C. V. 2008. Archetypal internet-scale source code searching. In Proceedings of the Working Group on Open Source Software (OSS) at the IFIP 20th World Computer Congress. B. Russo, Ed. Springer, New York, NY, 7."},{"key":"e_1_2_2_50_1","volume-title":"Proceedings of the 12th Text REtrieval Conference.","volume":"142","author":"Voorhees E.","year":"2003","unstructured":"Voorhees , E. 2003 . Overview of the trec 2003 question answering track . In Proceedings of the 12th Text REtrieval Conference. vol. 142 . NIST, Gaithersburg, MD. Voorhees, E. 2003. Overview of the trec 2003 question answering track. In Proceedings of the 12th Text REtrieval Conference. vol. 142. NIST, Gaithersburg, MD."},{"key":"e_1_2_2_51_1","doi-asserted-by":"publisher","DOI":"10.1002\/asi.v57:7"},{"key":"e_1_2_2_52_1","doi-asserted-by":"publisher","DOI":"10.1145\/581339.581402"},{"key":"e_1_2_2_53_1","doi-asserted-by":"publisher","DOI":"10.1145\/261640.261641"},{"key":"e_1_2_2_54_1","doi-asserted-by":"publisher","DOI":"10.1109\/TSE.2005.72"}],"container-title":["ACM Transactions on Software Engineering and Methodology"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2063239.2063243","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/2063239.2063243","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T09:54:25Z","timestamp":1750240465000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/2063239.2063243"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2011,12]]},"references-count":54,"journal-issue":{"issue":"1","published-print":{"date-parts":[[2011,12]]}},"alternative-id":["10.1145\/2063239.2063243"],"URL":"https:\/\/doi.org\/10.1145\/2063239.2063243","relation":{},"ISSN":["1049-331X","1557-7392"],"issn-type":[{"value":"1049-331X","type":"print"},{"value":"1557-7392","type":"electronic"}],"subject":[],"published":{"date-parts":[[2011,12]]},"assertion":[{"value":"2008-06-01","order":0,"name":"received","label":"Received","group":{"name":"publication_history","label":"Publication History"}},{"value":"2010-04-01","order":1,"name":"accepted","label":"Accepted","group":{"name":"publication_history","label":"Publication History"}},{"value":"2011-12-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}