Login
Create Account

Web Searching: A Quality Measurement Perspective

Lewandowski, Dirk and Höchstötter, Nadine . Web Searching: A Quality Measurement Perspective., 2007 In: Web Searching: Interdisciplinary Perspectives. Springer. (In Press) [Book chapter]

Preview

PDF
LewHoech_Preprint.pdf
Download (582kB) | Preview

English abstract

The purpose of this paper is to describe various quality measures for search engines and to ask whether these are suitable. We especially focus on user needs and their use of web search engines. The paper presents an extensive literature review and a first quality measurement model, as well. Findings include that search engine quality can not be measured by just retrieval effectiveness (the quality of the results), but should also consider index quality, the quality of the search features and search engine usability. For each of these sections, empirical results from studies conducted in the past, as well as from our own research are presented. These results have implications for the evaluation of search engines and for the development of better search systems that give the user the best possible search experience.

Item type:	Book chapter
Keywords:	search engines, quality measures, retrieval effetiveness, index quality
Subjects:	L. Information technology and library technology > LS. Search engines.
Depositing user:	Dirk Lewandowski
Date deposited:	23 May 2007
Last modified:	02 Oct 2014 12:07
URI:	http://hdl.handle.net/10760/9595

Check full metadata for this record

References

Acharya, A., Cutts, M., Dean, J., Haahr, P., Henzinger, M., Hoelzle, U., et al. (2005). Information Retrieval Based on Historical Data, USA.

Beitzel, S., Jensen, C., Chowdhury, A., Grossman, D., & Frieder, O. (2004). In Hourly Analysis of a Very Large Topically Categorized Web Query Log (pp. 321-328). Paper presented at the ACM SIGIR Conference on Re-search and Development in Information Retrieval, Sheffield, UK. ACM Press.

Bergman, M.K. (2001). The Deep Web: Surfacing Hidden Value. Journal of Elec-tronic Publishing, 7(1).

Bharat, K., & Broder, A. (1998). A Technique for Measuring the Relative Size and Overlap of Public Web Search Engines. Computer Networks and ISDN Systems, 30(1-7), 379-388.

Brin, S., & Page, L. (1998). The Anatomy of a Large-Scale Hypertextual Web Search Engine. Computer Networks and ISDN Systems, 30(1-7), 107-117.

Broder, A. (2002). A Taxonomy of Web Search. SIGIR Forum, 36(2), 3-10.

Broder, A., Kumar, R., Maghoul, F., Raghavan, P., Rajagopalan, S., Stata, R., et al. (2000). Graph Structure in the Web. Retrieved 15.4.2006, from http://www.almaden.ibm.com/webfountain/resources/GraphStructureintheWeb.pdf

Cacheda, F., & Viña, Á. (2001). Understanding How People Use Search Engines: A Statistical Analysis for E-Business (Vol. 1, pp. 319-325). Paper pre-sented at the e-2001 E-Business and E-Work Conference and Exhibition.

Ding, W., & Marchionini, G. (1996). A Comparative Study of Web Search Serv-ice Performance, Proceedings of the 59th American Society for Informa-tion Science Annual Meeting (pp. 136-142): Learned Information.

Ford, N., Miller, D., & Moss, N. (2002). Web Search Strategies and Retrieval Effectiveness: an Empirical Study. Journal of Documentation, 58(1), 30-48.

Geoghegan, T. (2004). Search Wars - Which is Best?, from news.bbc.co.uk/2/hi/uk_news/magazine/4003193.stm

Greisdorf, H., & Spink, A. (2001). Median Measure: An Approach to IR Systems Evaluation. Information Processing & Management, 37(6), 843-857.

Griesbaum, J. (2004). Evaluation of three German Search Engines: Altavista.de, Google.de and Lycos.de. Information Research, 9(4).

Griesbaum, J., Rittberger, M., & Bekavac, B. (2002). In: R. Hammwöhner, C. Wolff & C. Womser-Hacker (Eds.), Deutsche Suchmaschinen im Ver-gleich: AltaVista.de, Fireball.de, Google.de und Lycos.de (pp. 201-223). Paper presented at the Information und Mobilität. Optimierung und Ver-meidung von Mobilität durch Information. 8. Internationales Symposium für Informationswissenschaft. UVK.

Gulli, A., & Signorini, A. (2005). The Indexable Web is More Than 11.5 billion Pages (pp. 902-903). Paper presented at the Special Interest Tracks and Posters of the 14th International Conference on World Wide Web, Chiba, Japan.

Hoelscher, C., & Strube, G. (2000). Web Search Behavior of Internet Experts and Newbies (pp. 337-346). Paper presented at the 9th International World Wide Web Conference.

Ingwersen, P., & Järvelin, K. (2005). The Turn: Integration of Information Seek-ing and Retrieval in Context. Dordrecht: Springer.

Jansen, B. (2000). An Investigation Into the Use of Simple Queries on Web IR Systems. Information Research, An Electronic Journal, 6(1).

Jansen, B., & Spink, A. (2003). An Analysis of Web Documents Retrieved and Viewed (pp. 64-69). Paper presented at the 4th International Conference on Internet Computing.

Jansen, B., & Spink, A. (2006). How we are Searching the World Wide Web? A Comparision of Nine Searech Engine Transaction Logs. Information Processing and Management, 42(1), 248-263.

Ke, Y., Deng, L., Ng, W., & Lee, D.L. (2006). Web Dynamics and their Ramifica-tions for the Development of Web Search Engines. Computer Networks, 50(10), 1430-1447.

Kleinberg, J.M. (1999). Authoritative Sources in a Hyperlinked Environment. Journal of the ACM, 46, 604-632.

Korfhage, R.R. (1997). Information Storage and Retrieval. New York: Wiley.

Lawrence, S., & Giles, C.L. (1998). Searching the World Wide Web. Science, 280, 98-100.

Lawrence, S., & Giles, C.L. (1999). Accessibility of Information on the web. Na-ture, 400(8), 107-109.

Leighton, H.V., & Srivastava, J. (1999). First 20 Precision among World Wide Web Search Services (Search Engines). Journal of the American Society for Information Science, 50(10), 870-881.

Lewandowski, D. (2004a). Abfragesprachen und erweiterte Suchfunktionen von WWW-Suchmaschinen. Information Wissenschaft und Praxis, 55(2), 97-102.

Lewandowski, D. (2004b). Bewertung von linktopologischen Verfahren als bes-timmender Ranking-Faktor bei WWW-Suchmaschinen, Wissensorgani-sation und gesellschaftliche Verantwortung. 9. Tagung der Deutschen ISKO (Wissensorganisation'2004). Duisburg, Germany.

Lewandowski, D. (2004c). Date-restricted Queries in Web Search Engines. Online Information Review, 28(6), 420-427.

Lewandowski, D. (2005a). Web Searching, Search Engines and Information Re-trieval. Information Services and Use, 18(3), 137-147.

Lewandowski, D. (2005b). Yahoo - Zweifel an den Angaben zur Indexgröße, Suche in mehreren Sprachen. Password, 20(9), 21-22.

Lewandowski, D. (2006a). Aktualität als erfolgskritischer Faktor bei Suchmaschi-nen. Information Wissenschaft und Praxis, 57(3), 141-148.

Lewandowski, D. (2006b). Suchmaschinen als Konkurrenten der Bibliothekskata-loge: Wie Bibliotheken ihre Angebote durch Suchmaschinentechnologie attraktiver und durch Öffnung für die allgemeinen Suchmaschinen populärer machen können. Zeitschrift für Bibliothekswesen und Bibli-ographie, 53(2), 71-78.

Lewandowski, D. (2006c). Zur Bewertung der Qualität von Suchmaschinen. In: J. Eberspächer & S. Holtel (Eds.), Suchen und Finden im Internet (pp. 195-199). Heidelberg: Springer.

Lewandowski, D., & Mayr, P. (2006). Exploring the Academic Invisible Web. Li-brary Hi Tech, 24(4), 529-539.

Lewandowski, D., Wahlig, H., & Meyer-Bautor, G. (2006). The Freshness of Web search engine databases. Journal of Information Science, 32(2), 133-150.

MacCall, S.L., & Cleveland, A.D. (1999). A Relevance-based Quantitative Meas-ure for Internet Information Retrieval Evaluation (pp. 763-768). Paper presented at the Proceedings of the American Society for Information Science Annual Meeting.

Machill, M., Neuberger, C., Schweiger, W., & Wirth, W. (2003). Wegweiser im Netz: Qualität und Nutzung von Suchmaschinen. In M. Machill & C. Welp (Eds.), Wegweiser im Netz. Gütersloh: Bertelsmann Stiftung.

Machill, M., Neuberger, C., Schweiger, W., & Wirth, W. (2004). Navigating the Internet: A Study of German-Language Search Engines. European Jour-nal of Communication, 19(3), 321-347.

Notess, G.R. (2003). Search Engine Statistics: Freshness Showdown. Retrieved 4.1.2005, from http://www.searchengineshowdown.com/stats/freshness.shtml

Ntoulas, A., Cho, J., & Olston, C. (2004). What's New on the Web? The Evolution of the Web from a Search Engine Perspective. Paper presented at the Thirteenth WWW Conference, New York, USA.

Ozmutlu, H., Spink, A., & Ozmutlu, S. (2003). A Study of Multitasking Web Search (pp. 145-148). Paper presented at the International Conference on Information Technology: Computers and Communications.

Page, L., Brin, S., Motwani, R., & Winograd, T. (1998). The PageRank citation ranking: Bringing order to the Web. Retrieved 24.7.2006, from http://dbpubs.stanford.edu:8090/pub/1999-66

Parasuraman, A., Zeithaml, V.A., & Berry, L.L. (1988). SERVQUAL: A Multi-ple-item Scale for Measuring Consumer Perceptions of Service Quality. Journal of Retailing, 64(1), 12-40.

Risvik, K.M., & Michelsen, R. (2002). Search engines and Web dynamics. Com-puter Networks, 39(3), 289-302.

Saracevic, T. (1995). In Evaluation of Evaluation in Information Retrieval (pp. 138-146). Paper presented at the SIGIR'95, Seattle, CA. ACM Press.

Schmidt-Maenz (2007). Untersuchung des Suchverhaltens im Web - Interaktion von Internetnutzern mit Suchmaschinen, Dr. Kovac Verlag, Hamburg.

Schmidt-Maenz, N., & Bomhardt, C. (2005). Wie Suchen Onliner im Internet? Science Factory/Absatzwirtschaft(2), 5-8.

Schmidt-Maenz, N., & Gaul, W. (2005). Web Mining and Online Visibility. In C. Weihs & W. Gaul (Eds.), Classification - the Ubiquitous Challenge (pp. 418-425): Springer.

Schmidt-Maenz, N., & Koch, M. (2006). A General Classification of (Search) Queries and Terms (pp. 375-381). Paper presented at the 3rd Interna-tional Conference on Information Technologies: Next Generations, Las Vegas, Nevada, USA.

Sherman, C., & Price, G. (2001). The Invisible Web: Uncovering Information Sources Search Engines Can't See. Medford, NJ: Information Today.

Silverstein, C., Henzinger, M., Marais, H., & Moricz, M. (1999). Analysis of a Very Large Web Search Engine Query Log. ACM SIGIR Forum, 33(1), 6-12.

Singhal, A., & Kaszkiel, M. (2001). A case study in web search using TREC algo-rithms (pp. 708-716). Paper presented at the 10th international confer-ence on World Wide Web, Hong Kong.

Spink, A., & Jansen, B. (2004). Web Search: Public Searching of the Web (Vol. 6). Dordrecht, Boston, London: Kluwer Academic Publishers.

Spink, A., Jansen, B., & Ozmutlu, H. (2000). Use of Query Reformulation and Relevance Feedback by Excite Users. Internet Research: Electronic Net-working Applications and Policy, 19(4), 317-328.

Spink, A., Ozmutlu, S., Ozmutlu, H., & Jansen, B. (2002). U.S. Versus European Web Searching Processes. Journal of the American Society for Informa-tion Science and Technology, 53(8), 639-652.

Spink, A., Wolfram, D., Jansen, B., & Saracevic, T. (2001). Searching the Web: The Public and Their Queries. Journal of the American Society for In-formation Science and Technology, 52(3), 226-234.

Su, L.T. (1998). Value of Search Results as a Whole as the Best Single Measure of Information Retrieval Performance. Information Processing & Man-agement, 34(5), 557-579.

Sullivan, D. (2005). Search Engine Sizes. Retrieved 24.7.2006, from http://searchenginewatch.com/showPage.html?page=2156481

Vaughan, L. (2004). New Measurements for Search Engine Evaluation Proposed and Tested. Information Processing & Management, 40(4), 677-691.

Vaughan, L., & Thelwall, M. (2004). Search Engine Coverage Bias: Evidence and Possible Causes. Information Processing & Management, 40(4), 693-707.

Véronis, J. (2006). A Comparative Study of six Search Engines. Retrieved 15.3.2006, from http://www.up.univ-mrs.fr/veronis/pdf/2006-comparative-study.pdf

Wang, H., Xie, M., & Goh, T.N. (1999). Service Quality of Internet Search En-gines. Journal of Information Science, 25(6), 499-507.

Williams, M.E. (2005). The State of Databases Today: 2005. In Gale Directory of Databases (Vol. 2, pp. XV-XXV). Detroit, Mich.: Gale Group.

Wolff, C. (2000). Vergleichende Evaluierung von Such- und Metasuchmaschinen, 7. Internationales Symposium für Informationswissenschaft (ISI 2000) (pp. 31-38). Darmstadt, Germany: Universitätsverlag Konstanz.

Xie, M., Wang, H., & Goh, T.N. (1998). Quality Dimensions of Internet Search Engines. Journal of Information Science, 24(5), 365-372.

Zien, J., Meyer, J., Tomlin, J., & Liu, J. (2000). Web Query Characteristics and their Implications on Search Engines: Almaden Research Center.