{"status":"ok","message-type":"work","message-version":"1.0.0","message":{"indexed":{"date-parts":[[2025,6,19]],"date-time":"2025-06-19T04:38:22Z","timestamp":1750307902560,"version":"3.41.0"},"reference-count":23,"publisher":"Association for Computing Machinery (ACM)","issue":"2","license":[{"start":{"date-parts":[[2007,6,1]],"date-time":"2007-06-01T00:00:00Z","timestamp":1180656000000},"content-version":"vor","delay-in-days":0,"URL":"https:\/\/www.acm.org\/publications\/policies\/copyright_policy#Background"}],"content-domain":{"domain":["dl.acm.org"],"crossmark-restriction":true},"short-container-title":["SIGCSE Bull."],"published-print":{"date-parts":[[2007,6]]},"abstract":"<jats:p>In view of the recent progress in the field of internet search engines, there is a growing need for mechanisms to evaluate the performance of these useful and popular tools. So far, the vast majority of researchers have relied on the information-retrieval metrics of \"precision\" and \"recall\" that quantify the occurrence of \"hits\" and \"misses\" in the returned list of documents. What they fail to do is to measure the quality of the ranking that the search engine has provided. This paper wants to rectify the situation. We discuss the issue in some detail, and then propose a new mechanism that we believe is better suited for our needs.<\/jats:p>","DOI":"10.1145\/1272848.1272894","type":"journal-article","created":{"date-parts":[[2007,9,14]],"date-time":"2007-09-14T13:44:55Z","timestamp":1189777495000},"page":"97-101","update-policy":"https:\/\/doi.org\/10.1145\/crossmark-policy","source":"Crossref","is-referenced-by-count":2,"title":["Search engine ranking efficiency evaluation tool"],"prefix":"10.1145","volume":"39","author":[{"given":"Wadee S.","family":"Alhalabi","sequence":"first","affiliation":[{"name":"University of Miami, Miami, FL"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Miroslav","family":"Kubat","sequence":"additional","affiliation":[{"name":"University of Miami, Coral Gables, FL"}],"role":[{"role":"author","vocabulary":"crossref"}]},{"given":"Moiez","family":"Tapia","sequence":"additional","affiliation":[{"name":"University of Miami, Coral Gables, FL"}],"role":[{"role":"author","vocabulary":"crossref"}]}],"member":"320","published-online":{"date-parts":[[2007,6]]},"reference":[{"key":"e_1_2_1_1_1","doi-asserted-by":"publisher","DOI":"10.5555\/347319.346332"},{"key":"e_1_2_1_2_1","volume-title":"Precision and Recall of Five Search Engines for Retrieval of Scholarly Information in the Field of Biotechnology. Webology","volume":"2","author":"Shafi S. M.","year":"2005","unstructured":"S. M. Shafi and Rafiq A . Rather . Precision and Recall of Five Search Engines for Retrieval of Scholarly Information in the Field of Biotechnology. Webology , Volume 2 , Number 2, August , 2005 S. M. Shafi and Rafiq A. Rather. Precision and Recall of Five Search Engines for Retrieval of Scholarly Information in the Field of Biotechnology. Webology, Volume 2, Number 2, August, 2005"},{"key":"e_1_2_1_3_1","volume-title":"Chu and Marilyn Rosenthal. Search Engines for the World Wide Web: A Comparative Study and Evaluation Methodology. ASIS 1996 Annual Conference Proceedings","author":"Heting","year":"1996","unstructured":"Heting Chu and Marilyn Rosenthal. Search Engines for the World Wide Web: A Comparative Study and Evaluation Methodology. ASIS 1996 Annual Conference Proceedings , Octobr 1996 . Heting Chu and Marilyn Rosenthal. Search Engines for the World Wide Web: A Comparative Study and Evaluation Methodology. ASIS 1996 Annual Conference Proceedings, Octobr 1996."},{"key":"e_1_2_1_4_1","doi-asserted-by":"publisher","DOI":"10.1007\/BF00125371"},{"key":"e_1_2_1_5_1","doi-asserted-by":"publisher","DOI":"10.1145\/333135.333137"},{"key":"e_1_2_1_6_1","doi-asserted-by":"publisher","DOI":"10.1145\/944012.944015"},{"key":"e_1_2_1_7_1","volume-title":"Oscar Miranda and Silvio Meira. Measuring the Effect of Centroid Size on Web Search Precision and Recall. In Proceedings 8th Annual Conference of the Internet Society (INET'98)","author":"Goncalves Pedro","year":"1998","unstructured":"Pedro Goncalves , Jacques Robin , Thiago Santos , Oscar Miranda and Silvio Meira. Measuring the Effect of Centroid Size on Web Search Precision and Recall. In Proceedings 8th Annual Conference of the Internet Society (INET'98) . Geneva, Switzerland , July , 1998 . http:\/\/www.isoc.org\/inet98\/proceedings\/1x\/1x_8.htm. Pedro Goncalves, Jacques Robin, Thiago Santos, Oscar Miranda and Silvio Meira. Measuring the Effect of Centroid Size on Web Search Precision and Recall. In Proceedings 8th Annual Conference of the Internet Society (INET'98). Geneva, Switzerland, July, 1998. http:\/\/www.isoc.org\/inet98\/proceedings\/1x\/1x_8.htm."},{"key":"e_1_2_1_8_1","volume-title":"Jerry H. Ying and Dik L. Lee. A World Wide Web Resource Discovery System. Proceedings of the Fourth International World Wide Web Conference","author":"Yuwono Budi","year":"1995","unstructured":"Budi Yuwono , Savio L. Y. Lam , Jerry H. Ying and Dik L. Lee. A World Wide Web Resource Discovery System. Proceedings of the Fourth International World Wide Web Conference , Boston, MA. , Dec. 1995 . Budi Yuwono, Savio L. Y. Lam, Jerry H. Ying and Dik L. Lee. A World Wide Web Resource Discovery System. Proceedings of the Fourth International World Wide Web Conference, Boston, MA., Dec. 1995."},{"volume-title":"Also in Proceedings of the 8th International World Wide Web Conference.","author":"Hawking David","key":"e_1_2_1_9_1","unstructured":"David Hawking , Nick Craswell , Paul Thistlewaite and Donna Harman . Results and Challenges in Web Search Evaluation. Computer Networks, 31(11--16): 1321--1330, May 1999 . Also in Proceedings of the 8th International World Wide Web Conference. David Hawking, Nick Craswell, Paul Thistlewaite and Donna Harman. Results and Challenges in Web Search Evaluation. Computer Networks, 31(11--16): 1321--1330, May 1999. Also in Proceedings of the 8th International World Wide Web Conference."},{"key":"e_1_2_1_10_1","first-page":"107","volume-title":"Ashman and Thistlewaite {13}","author":"Brin Sergey","unstructured":"Sergey Brin and Lawrence Page . The anatomy of a large-scale hypertextual Web search engine . In Ashman and Thistlewaite {13} , pages 107 -- 117 . Brisbane , Australia . Sergey Brin and Lawrence Page. The anatomy of a large-scale hypertextual Web search engine. In Ashman and Thistlewaite {13}, pages 107--117. Brisbane, Australia."},{"key":"e_1_2_1_12_1","first-page":"95","volume-title":"Ashman and Thistlewaite {13}","author":"Lawrence Steve","unstructured":"Steve Lawrence and C. Lee Giles . Inquirus , the NECI meta search engine . In Ashman and Thistlewaite {13} , pages 95 -- 105 . Brisbane , Australia . Steve Lawrence and C. Lee Giles. Inquirus, the NECI meta search engine. In Ashman and Thistlewaite {13}, pages 95--105. Brisbane, Australia."},{"key":"e_1_2_1_13_1","volume-title":"Proceedings of the Seventh International World Wide Web Conference","volume":"30","author":"Ashman Helen","year":"1998","unstructured":"Helen Ashman and Paul Thistlewaite , editors. Proceedings of the Seventh International World Wide Web Conference , volume 30 of Computer Networks and ISDN Systems. The International Journal of Computer and Telecommunications Networking, Amsterdam , April 1998 . Elsevier. Brisbane, Australia. Helen Ashman and Paul Thistlewaite, editors. Proceedings of the Seventh International World Wide Web Conference, volume 30 of Computer Networks and ISDN Systems. The International Journal of Computer and Telecommunications Networking, Amsterdam, April 1998. Elsevier. Brisbane, Australia."},{"key":"e_1_2_1_14_1","first-page":"164","volume-title":"12th International Conference on Data Engineering (ICDE'96)","author":"Yuwono B.","unstructured":"B. Yuwono and D. L. Lee . Search and ranking algorithms for locating resources on the World Wide Web . 12th International Conference on Data Engineering (ICDE'96) p. 164 B. Yuwono and D. L. Lee. Search and ranking algorithms for locating resources on the World Wide Web. 12th International Conference on Data Engineering (ICDE'96) p. 164"},{"key":"e_1_2_1_15_1","doi-asserted-by":"publisher","DOI":"10.1145\/1013367.1013459"},{"key":"e_1_2_1_16_1","volume-title":"Trotman and Richard A. O'Keefe. Identifying and Ranking Relevant Document Elements. In INEX '03","author":"Andrew","year":"2003","unstructured":"Andrew Trotman and Richard A. O'Keefe. Identifying and Ranking Relevant Document Elements. In INEX '03 , 2003 . Andrew Trotman and Richard A. O'Keefe. Identifying and Ranking Relevant Document Elements. In INEX '03, 2003."},{"issue":"3","key":"e_1_2_1_17_1","first-page":"87","article-title":"Relevancy Rankings --- Pay for Performance Search Engines in the Hot Seat","volume":"27","author":"Goh Dion H.","unstructured":"Dion H. Goh and Rebecca P. Ang . Relevancy Rankings --- Pay for Performance Search Engines in the Hot Seat . Online Information Review , 27 ( 3 ), 87 -- 93 . Dion H. Goh and Rebecca P. Ang. Relevancy Rankings --- Pay for Performance Search Engines in the Hot Seat. Online Information Review, 27(3), 87--93.","journal-title":"Online Information Review"},{"key":"e_1_2_1_18_1","first-page":"31","volume-title":"Numerical Solution of Markov Chains","author":"Kamvar Sepandar","unstructured":"Sepandar Kamvar , Taher Haveliwala and Gene Golub . Adaptive Methods for the Computation of PageRank . Numerical Solution of Markov Chains , pages 31 -- 44 . Sepandar Kamvar, Taher Haveliwala and Gene Golub. Adaptive Methods for the Computation of PageRank. Numerical Solution of Markov Chains, pages 31--44."},{"key":"e_1_2_1_19_1","unstructured":"Miguel Costa and Mario J. Silva. Optimizing Ranking Calculation in Web Search Engines: a Case Study.  Miguel Costa and Mario J. Silva. Optimizing Ranking Calculation in Web Search Engines: a Case Study."},{"volume-title":"Ljosland. Evaluation of Web Search Engine and the Search for Better Ranking Algorithms. SIGIR99 Workshop on Evaluation of Web Retrieval (1999)","author":"Mildrid","key":"e_1_2_1_20_1","unstructured":"Mildrid Ljosland. Evaluation of Web Search Engine and the Search for Better Ranking Algorithms. SIGIR99 Workshop on Evaluation of Web Retrieval (1999) Mildrid Ljosland. Evaluation of Web Search Engine and the Search for Better Ranking Algorithms. SIGIR99 Workshop on Evaluation of Web Retrieval (1999)"},{"key":"e_1_2_1_21_1","doi-asserted-by":"publisher","DOI":"10.1145\/564376.564389"},{"volume-title":"Bifet and Carlos Catillo. An Analysis of Factors Used in Search Engine Ranking. Proceedings of the 1st International Workshop on Adversarial Information Retrieval on the Web (AIRWeb).","author":"Albert","key":"e_1_2_1_22_1","unstructured":"Albert Bifet and Carlos Catillo. An Analysis of Factors Used in Search Engine Ranking. Proceedings of the 1st International Workshop on Adversarial Information Retrieval on the Web (AIRWeb). Albert Bifet and Carlos Catillo. An Analysis of Factors Used in Search Engine Ranking. Proceedings of the 1st International Workshop on Adversarial Information Retrieval on the Web (AIRWeb)."},{"key":"e_1_2_1_23_1","doi-asserted-by":"publisher","DOI":"10.1002\/(SICI)1097-4571(1999)50:10%3C882::AID-ASI5%3E3.0.CO;2-6"},{"key":"e_1_2_1_24_1","doi-asserted-by":"publisher","DOI":"10.1109\/TKDE.2003.1208999"}],"container-title":["ACM SIGCSE Bulletin"],"original-title":[],"language":"en","link":[{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1272848.1272894","content-type":"unspecified","content-version":"vor","intended-application":"text-mining"},{"URL":"https:\/\/dl.acm.org\/doi\/pdf\/10.1145\/1272848.1272894","content-type":"unspecified","content-version":"vor","intended-application":"similarity-checking"}],"deposited":{"date-parts":[[2025,6,18]],"date-time":"2025-06-18T14:47:36Z","timestamp":1750258056000},"score":1,"resource":{"primary":{"URL":"https:\/\/dl.acm.org\/doi\/10.1145\/1272848.1272894"}},"subtitle":[],"short-title":[],"issued":{"date-parts":[[2007,6]]},"references-count":23,"journal-issue":{"issue":"2","published-print":{"date-parts":[[2007,6]]}},"alternative-id":["10.1145\/1272848.1272894"],"URL":"https:\/\/doi.org\/10.1145\/1272848.1272894","relation":{},"ISSN":["0097-8418"],"issn-type":[{"type":"print","value":"0097-8418"}],"subject":[],"published":{"date-parts":[[2007,6]]},"assertion":[{"value":"2007-06-01","order":2,"name":"published","label":"Published","group":{"name":"publication_history","label":"Publication History"}}]}}