The retrieval effectiveness of search engines on navigational queries

Lewandowski, Dirk The retrieval effectiveness of search engines on navigational queries., 2011 [Preprint]

[thumbnail of ASLIB2011_preprint.pdf]
Preview
 PDF
ASLIB2011_preprint.pdf
Download (415kB) | Preview

English abstract

Purpose – To test major Web search engines on their performance on navigational queries, i.e. searches for homepages. Design/methodology/approach – 100 real user queries are posed to six search engines (Google, Yahoo, MSN, Ask, Seekport, and Exalead). Users described the desired pages, and the results position of these is recorded. Measured success N and mean reciprocal rank are calculated. Findings – Performance of the major search engines Google, Yahoo, and MSN is best, with around 90 percent of queries answered correctly. Ask and Exalead perform worse but receive good scores as well. Research limitations/implications – All queries were in German, and the German-language interfaces of the search engines were used. Therefore, the results are only valid for German queries. Practical implications – When designing a search engine to compete with the major search engines, care should be taken on the performance on navigational queries. Users can be influenced easily in their quality ratings of search engines based on this performance. Originality/value – This study systematically compares the major search engines on navigational queries and compares the findings with studies on the retrieval effectiveness of the engines on informational queries.

Item type: Preprint
Keywords: retrieval effectiveness, search engine, navigational queries
Subjects: L. Information technology and library technology > LC. Internet, including WWW.
L. Information technology and library technology > LS. Search engines.
Depositing user: Dirk Lewandowski
Date deposited: 27 Jun 2012
Last modified: 02 Oct 2014 12:22
URI: http://hdl.handle.net/10760/17233

References

Broder, A. (2002), “A taxonomy of Web search”, SIGIR Forum, Vol. 36 No. 2, pp. 3-10.

Craswell, N., and Hawking, D. (2005), “Overview of the TREC-2004 Web Track” available at: trec.nist.gov/pubs/trec12/papers/WEB.OVERVIEW.pdf (accessed 17 July 2008).

Frants, V.I., Shapiro, J., and Voiskunskii, V.G. (1997), Automated Information Retrieval: Theory and Methods, Academic Press, San Diego.

Granka, L.A., Joachims, T. and Gay, G. (2004), “Eye-tracking analysis of user behavior in WWW search”, Proceedings of Sheffield SIGIR, Twenty-Seventh Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 478-479.

Griesbaum, J. (2004), “Evaluation of three German search engines: Altavista.de, Google.de and Lycos.de”, Information Research, Vol. 9 No.4.

Hawking, D. and Craswell, N. (2005), “The Very Large Collection and Web Tracks”, in Voorhees, E.M. and Harman, D.K. (Eds.), TREC Experiment and Evaluation in Information Retrieval, MIT Press, Cambridge, Massachusetts, pp. 199-231.

Jansen, B.J., Booth, D.L., and Spink, A. (2008), “Determining the informational, navigational, and transactional intent of Web queries”, Information Processing & Management, Vol.44 No.3, pp. 1251-1266.

Kang, I.H., and Kim, G. (2003), “Query Type Classification for Web Document Retrieval”, SIGIR Forum, ACM Special Interest Group on Information Retrieval, SPEC. ISS., pp. 64-71.

Keane, M.T., O'Brien, M. and Smyth, B. (2008), “Are people biased in their use of search engines?” Communications of the ACM, Vol. 51 No. 2, pp. 49-52.

Kilgour, F.G. (1999), “Retrieval effectiveness of surname-title-word searches for known items by academic library users”, Journal of the American Society for Information Science, Vol. 50 No. 2-3, pp. 265-270.

Lewandowski, D. (2006), “Query types and search topics of German Web search engine users”, Information Services & Use, Vol. 26 No. 4, pp. 261-269.

Lewandowski, D. (2008), “The Retrieval Effectiveness of Web Search Engines: Considering Results Descriptions”, Journal of Documentation, Vol. 64 [to appear].

Lewandowski, D. and Höchstötter, N. (2008), “Web Searching: A Quality Measurement Perspective”. in Spink, A.; Zimmer, M. (Eds.), Web Search: Multidisciplinary Perspectives, Springer, Berlin, pp. 309-340.

MacFarlane, A. (2007), “Evaluation of Web search for the information practitioner”, Aslib Proceedings: New Information Perspectives, Vol. 59 No. 4-5, pp. 352-366.

Machill, M. and Neuberger, C., Schweiger, W., & Wirth, W. (2003). “Wegweiser im Netz: Qualität und Nutzung von Suchmaschinen“, in Machill, M. and Welp, C. (Eds.), Wegweiser im Netz, Bertelsmann Stiftung, Gütersloh, pp. 13-490

Rose, D.E. and Levinson, D. (2004), “Understanding user goals in Web search”, Thirteenth International World Wide Web Conference Proceedings, WWW2004.

Schmidt-Maenz, N. and Bomhardt, C. (2005), “Wie suchen Onliner im Internet?“, Science Factory/Absatzwirtschaft, 2 , pp. 5-8.

“Suchmaschinen-Marktanteile (2008)”, available at: www.luna-park.de/home/internet-fakten/suchmaschinenmarktanteile.html (accessed 17 July 2008).

Véronis, J. (2006), “A comparative study of six search engines”, available at: www.up.univmrs.fr/veronis/pdf/2006-comparative-study.pdf (accessed 17 July 2008).

“Web-Barometer”, available at: www.webhits.de/deutsch/index.shtml?webstats.html (accessed 17 July 2008).


Downloads

Downloads per month over past year

Actions (login required)

View Item View Item
Contact us
Send email Sponsors
Sponsor: AIMS Sponsor: D&P Sponsor: UniNa CAB Get connected
Facebook Facebook Twitter Twitter Linkedin LinkedIn RSS RSS