E-LIS, Eprints in Library and Information Science Homepage E-LIS, Eprints in Library and Information Science
   home   |   about   |   search   |   browse   |   register   |   registered users area   |   help   |   FAQ   |   JITA   

The Freshness of Web search engines’ databases

Lewandowski, Dirk and Wahlig, Henry and Meyer-Bautor, Gunnar (2005) The Freshness of Web search engines’ databases. Journal of Information Science 31.

Full text available as:
PDF - Requires Adobe Acrobat Reader or other PDF viewer.

Abstract

This study measures the frequency in which search engines update their indices. Therefore, 38 websites that are updated on a daily basis were analysed within a time-span of six weeks. The analysed search engines were Google, Yahoo and MSN. We find that Google performs best overall with the most pages updated on a daily basis, but only MSN is able to update all pages within a time-span of less than 20 days. Both other engines have outliers that are quite older. In terms of indexing patterns, we find different approaches at the different engines: While MSN shows clear update patterns, Google shows some outliers and the update process of the Yahoo index seems to be quite chaotic. Implications are that the quality of different search engine indices varies and not only one engine should be used when searching for current content.

Keywords:search engines; Online Information Retrieval; World Wide Web; index quality; index freshness
Subjects:L. Information technology and library technology. > LS. Search engines.
ID Code:4619
Deposited By:Lewandowski, Dirk
Deposited On:07 September 2005
Alternative Locations:http://www.durchdenken.de/lewandowski/doc/jis2005.php
All fields:Show all fields

A. Acharya, A., M. Cutts, J. Dean, P. Haahr, M. Henzinger, U. Hoelzle, S. Lawrence, K. Pfleger, O. Sercinoglu, and S. Tong, Information retrieval based on historical data (Patent Application US 2005/0071741 A1, 2005)

V. Cothey, Web-Crawling Reliability, Journal of the American Society for Information Science and Technology 55(14) (2004) 1228-1238.

N. Ford, D. Miller and N. Moss, Web search strategies and retrieval effectiveness: an empirical study, Journal of Documentation 58(1) (2002) 30-48

R. Fries, W. Schweibenz, J. Strobel and P. Wiland, Was indexieren Suchmaschinen? Eine Untersuchung zu Indexierungsmechanismen von Suchmaschinen im World Wide Web, BIT Online 4(1) (2001) 49-56.

J. Griesbaum, Evaluation of three German search engines: Altavista.de, Google.de and Lycos.de (2004). Available at: http://informationr.net/ir/9-4/paper189.html (accessed 8 May 2005).

J. Griesbaum, M. Rittberger and B. Bekavac, Deutsche Suchmaschinen im Vergleich: AltaVista.de, Fireball.de, Google.de und Lycos.de. In: R. Hammwöhner, C. Wolff, C. Womser-Hacker (eds.), Information und Mobilität. Optimierung und Vermeidung von Mobilität durch Information. Proceedings des 8. Internationalen Symposiums für Informationswissenschaft (UVK, Konstanz, 2002).

S. Lawrence and C.L. Giles, Searching the World Wide Web, Science 280 (1998) 98-100.

S. Lawrence and C.L. Giles: Accessibility of information on the web. Nature 400(8) (1999) 107-109.

H. Leighton and J. Srivastava, First 20 Precision among World Wide Web Search Services (Search Engines), Journal of the American Society for Information Science 50(10) (1999) 870-881.

D. Lewandowski, Date-restricted queries in web search engines, Online Information Review 28(6) (2004) 420-427.

L. Lo Grasso and H. Wahlig, Google und seine Suchparameter: Eine Top 20-Precision Analyse anhand repräsentativ ausgewählter Anfragen. Information Wissenschaft und Praxis 56(2) (2005) 77-86.

M. Machill and C. Welp (eds.), Wegweiser im Netz: Qualität und Nutzung von Suchmaschinen (Verlag Bertelsmann Stiftung, Gütersloh, 2003).

M. Machill, D. Lewandowski and S. Karzauninkat, Journalistische Aktualität im Internet. Ein Experiment mit den “News-Suchfunktionen” von Suchmaschinen. In: M. Machill and N. Schneider (eds.), Suchmaschinen: Eine Herausforderung für die Medienpolitik, (Vistas, Berlin, 2005).

A. Mowshowitz and A. Kawaguchi, Assessing bias in search engines, Information Processing & Management 38(1) (2001) 141-156.

G. Notess, Search Engine Statistics: Freshness Showdown [Data from 17 May 2003] (2003). Available at: http://www.searchengineshowdown.com/stats/freshness.shtml (accessed 17 April 2005).

G. Notess, Search Engine Statistics: Freshness Showdown [Data from 20 October 2002] (2002). Available at: http://www.searchengineshowdown.com/stats/0210freshness.shtml (accessed 17 April 2005).

G. Notess, Search Engine Statistics: Freshness Showdown [Data from4 April 2002] (2002). Available at: http://www.searchengineshowdown.com/stats/0204freshness.shtml (accessed 17 April 2005).

G. Notess, Search Engine Statistics: Freshness Showdown [Data from 7 March 2002] (2002). Available at: http://www.searchengineshowdown.com/stats/0203freshness.shtml (accessed 17 April 2005).

G. Notess, Search Engine Statistics: Freshness Showdown [Data from 13 August 2001] (2001). Available at: http://www.searchengineshowdown.com/stats/0108freshness.shtml (accessed 17 April 2005).

A. Ntoulas, J. Cho and C. Olston, What's New on the Web? The Evolution of the Web from a Search Engine Perspective (2004). In: Proceedings of the Thirteenth WWW Conference, New York, USA. http://oak.cs.ucla.edu/~ntoulas/pubs/ntoulas_new.pdf (accessed 8 May 2005).

A. Singhal, and M. Kaszkiel, A Case Study in Web Search using TREC Algorithms. In: Tenth World Wide Web Conference 2001: Proceedings of the 10th World Wide Web Conference (ACM Press, New York, 2001).

D. Sullivan: Nielsen Net Ratings Search Engine Ratings, Searchenginewatch.com. http://searchenginewatch.com/reports/article.php/2156451 (accessed 22 April 2005).

L. Vaughan and M. Thelwall, Search Engine Coverage Bias: Evidence and Possible Causes, Information Processing & Management 40(4) (2004) 693-707.

C. Wolff, Effektivität von Recherchen im WWW: Vergleichende Evaluierung von Such- und Metasuchmaschinen. In: G. Knorz and R. Kuhlen (eds.), Informationskompetenz - Basiskompetenz in der Informationsgesellschaft, Proceedings des 7. Internationalen Symposiums für Informationswissenschaft (UVK, Konstanz, 2000).

Archive Staff Only: edit this record

   home   |   about   |   search   |   browse   |   register   |   registered users area   |   help   |   FAQ   |   JITA   

Send comments and questions to      |     E-LIS is hosted by AEPIC at CILEA      |     Powered by GNU Eprints software