Evaluation of three German search engines: Altavista.de, Google.de and Lycos.de

Griesbaum, Joachim Evaluation of three German search engines: Altavista.de, Google.de and Lycos.de. Information Research, 2004, vol. 9, n. 4. [Journal article (Unpaginated)]

[thumbnail of paper189.html] HTML
paper189.html
Download (98kB)

English abstract

The goal of this study was to investigate the retrieval effectiveness of three popular German Web search services. For this purpose the engines Altavista.de, Google.de and Lycos.de were compared with each other in terms of the precision of their top twenty results. The test panelists were based on a collection of fifty randomly selected queries, and relevance assessments were made by independent jurors. Relevance assessments were acquired separately a) for the search results themselves and b) for the result descriptions on the search engine results pages. The basic findings were: 1.) Google reached the best result values. Statistical validation showed that Google performed significantly better than Altavista, but there was no significant difference between Google and Lycos. Lycos also attained better values than Altavista, but again the differences reached no significant value. In terms of top twenty precision, the experiment showed similar outcomes to the preceding retrieval test in 2002. Google, followed by Lycos and then Altavista, still performs best, but the gaps between the engines are closer now. 2.) There are big deviations between the relevance assignments based on the judgement of the results themselves and those based on the judgements of the result descriptions on the search engine results pages.

Item type: Journal article (Unpaginated)
Keywords: search engine evaluation
Subjects: L. Information technology and library technology
Depositing user: Joachim Griesbaum
Date deposited: 15 Dec 2004
Last modified: 02 Oct 2014 11:59
URI: http://hdl.handle.net/10760/5746

References

Bar-Ilan, J. (2002). Methods for measuring search engine performance over time. Journal of the American Society for Information Science and Technology, 53(4), 308-319.

Buckley, C. & Voorhees, E.M. (2000). Evaluating Evaluation Measure Stability. In N. J. Belkin, P. Ingwersen, M.-K.Leong and E. Yannakoudakis (Eds.); Proceedings of SIGIR'00, (pp. 33-40). New York: ACM Press.

Cleverdon, C. W., Mills, J. & Keen, M. (1966). Factors determining the performance of indexing systems, Vol. 1: Design, Vol. 2: Test results. Cranfield, UK: College of Aeronautics.

Dennis, S., Bruza, P. & McArthur, R. (2002). Web searching: a process-oriented experimental study of three interactive search paradigms. Journal of the American Society for Information Science and Technology, 53(2), 120-133.

Eguchi, K., Oyama, K., Ishida, E., Kando, N. & Kuriyama, K. (2002). Overview of the Web retrieval task at the Third NTCIR Workshop. Proceedings of the Third NTCIR Workshop on research in information Retrieval, Automatic Text Summarization and Question Answering (September 2001-October, 2002), Tokyo: National Institute of Informatics (NII). Retrieved 27 April, 2003 from http://research.nii.ac.jp/ntcir/workshop/OnlineProceedings3/NTCIR3-OV-WEB-EguchiK.pdf.

Ford, N., Miller, D. & Moss, N. (2001). The role of individual differences in Internet searching: an empirical study. Journal of the American Society for Information Science and Technology, 52(12), 1049-1066.

Gordon, M. and Pathak, P. (1999). Finding information on the World Wide Web: the retrieval effectiveness of search engines. Information Processing & Management, 35(2), 141-180.

Griesbaum, J. (2000). Evaluierung hybrider Suchsysteme im WWW. Unpublished diploma thesis, Universität Konstanz, Konstanz, Germany. Retrieved 28 April, 2003 from http://www.inf.uni-konstanz.de/%7Egriesbau/files/evaluierung_hybrider_suchsysteme_im_www.pdf.

Griesbaum, J., Rittberger, M. & Bekavac, B. (2002). Deutsche Suchmaschinen im Vergleich: AltaVista.de, Fireball.de, Google.de und Lycos.de. In R. Hammwöhner, C. Wolff, and C. Womser-Hacker (Eds.); Information und Mobilität, Optimierung und Vermeidung von Mobilität durch Information, Proceedings des 8. Internationalen Symposiums für Informationswissenschaft, (pp.201-223). Konstanz: UVK, 201-223.

Gurrin, C. & Smeaton, A. (2003). Improving the evaluation of Web search systems. In F. Sebastiani (Ed.), Advances in information retrieval: 25th European Conference on IR Research, ECIR 2003, Pisa, Italy, April 14-16, 2003. Proceedings, (pp.25-40) Berlin; New York: Springer. (Lecture Notes in Computer Science 2633)

Harter, S.P. (1996). Variations in relevance assessments and the measurement of retrieval effectiveness. Journal of the American Society for Information Science, 47(1), 37-49.

Hawking, D., Craswell, N., Bailey, P. & Griffiths, K. (2001). Measuring search engine quality. Journal of Information Retrieval, 4(1), 33-59.

Inktomi Corp. Web search relevance test. (2003). Retrieved 6 May, 2003 from the Veritest Website at http://www.veritest.com/clients/reports/inktomi/inktomi_Web_search_test.pdf

Jansen, B., Spink, A. & Saracevic, T. (2000). Real life, real users, and real needs: a study and analysis of user queries on the Web. Information Processing & Management, 36(2), 207-227.

Kowalski, G. (1997). Information retrieval systems, theory and implementation. Boston, MA: Kluwer Academic Publishers

Leighton, H. V. & Srivastava, J. (1999). First 20 precision among World Wide Web search services (search engines). Journal of the American Society for Information Science, 50(10), 870-881.

Lesk, M. (1995). The seven ages of information retrieval. In Proceedings of the Conference for the 50th anniversary of As We May Think. (pp. 12-14). Cambridge, MA: MIT Press.

Mandl, T. (2003). Web- und Multimedia-Dokumente. Neuere Entwicklungen bei der Evaluierung von Information Retrieval Systemen. Information: Wissenschaft & Praxis, 54(4), 203-210.

Robertson, S.E. (1981). The methodology of information retrieval experiments. In K.S. Jones, (Ed.); Information retrieval experiment. (pp.9-31) London: Butterworth. 9-31.

Schamber, L. (1994). Relevance and information behavior. Annual Review of Information Science and Technology. 29,, 3-48.

Shang, Yi. & Longzhuang, Li. (2002). Precision evaluation of search engines. World Wide Web, 5(2), 159-179.

Siegel, S. (1987). Nichtparametrische statistische Methoden (3rd ed.). Eschborn bei Frankfurt am Main: Klotz

Spink, A. (2002). A user centered approach to evaluating human interaction with Web search engines: an exploratory study. Information Processing & Management, 38(3), 410-426.

Sullivan, D. (2002, December 5). In search of the relevancy figure. SearchEngineWatch.com Retrieved 24 April, 2003 from http://www.searchenginewatch.com/sereport/article.php/2165151

Sullivan, D. (2002, April 29). Jupiter MMXI European search engine ratings. SearchEngineWatch.com Retrieved 19 April 2004 from http://Web.archive.org/Web/20030618111440/ http://www.searchenginewatch.com/reports/article.php/2156441)

Sullivan, D. (2004, April 28). Major search engines and directories. SearchEngineWatch.com Retrieved 24 April, 2003 from http://www.searchenginewatch.com/links/article.php/2156221

Sullivan, D. (n.d.). Search engine optimization & marketing glossary. Palo Alto, CA: SEMPO: Search Engine Marketing Professional Organization. Retrieved 19 April, 2004 from the SEMPO Website at http://www.sempo.org/search-engine-marketing-glossary.php

Tague-Sutcliffe, J. (1992). The pragmatics of information retrieval experimentation, revisited. Information Processing & Management, 28(4), 467-490.

Womser-Hacker, C. (1989).Der PADOK Retrievaltest. Zur Methode und Verwendung statistischer Verfahren bei der Bewertung von Information- Retrieval-Systemen. Hildesheim Georg Olms.


Downloads

Downloads per month over past year

Actions (login required)

View Item View Item
Contact us
Send email Sponsors
Sponsor: AIMS Sponsor: D&P Sponsor: UniNa CAB Get connected
Facebook Facebook Twitter Twitter Linkedin LinkedIn RSS RSS