|
|
E-LIS. E-prints in Library and Information Science >
List of countries by continent >
EUROPE >
Germany >
Journal Article (On-line/Unpaginated) >
Lewandowski, D. The Retrieval Effectiveness of Web Search Engines: Considering
Results Descriptions, 2008. In Journal of Documentation. Emerald. (In Press) [Journal Article (On-line/Unpaginated)].
See the references list of this item
Citable URI:
http://hdl.handle.net/10760/11258
Files in This Item:
| File |
Description |
Size | Format | Visibility |
|---|
| JDoc2008_Preprint.pdf | | 1.01 MB | Adobe PDF | View/Open
|
|
| Author(s): | Lewandowski, Dirk |
| Title: | The Retrieval Effectiveness of Web Search Engines: Considering
Results Descriptions |
| Subjects: | L. Information technology and library technology > LS. Search engines |
| Date: | 2008 |
| Abstract: | Purpose: To compare five major Web search engines (Google, Yahoo, MSN, Ask.com, and Seekport) for their retrieval
effectiveness, taking into account not only the results but also the results descriptions.
Design/Methodology/Approach: The study uses real-life queries. Results are made anonymous and are randomised. Results
are judged by the persons posing the original queries.
Findings: The two major search engines, Google and Yahoo, perform best, and there are no significant differences between
them. Google delivers significantly more relevant result descriptions than any other search engine. This could be one reason
for users perceiving this engine as superior.
Research Limitations: The study is based on a user model where the user takes into account a certain amount of results rather
systematically. This may not be the case in real life.
Practical Implications: Implies that search engines should focus on relevant descriptions. Searchers are advised to use other
search engines in addition to Google.
Originality/Value: This is the first major study comparing results and descriptions systematically and proposes new retrieval
measures to take into account results descriptions.
Article type: Research paper |
| Publication: | Journal of Documentation |
| Volume: | 64 |
| Publisher: | Emerald |
| Alternative Locations: | http://www.durchdenken.de/lewandowski/doc/JDoc2008_preprint.pdf |
| Keywords: | Word Wide Web / search engines / retrieval effectiveness / results descriptions / retrieval measures |
| Country: | Germany |
| Type: | Journal Article (On-line/Unpaginated) |
| Rights: | http://eprints.rclis.org/copyright/ |
|
References
- Borlund, P. (2003). The concept of relevance in IR. Journal of the American Society for Information Science and Technology, Vol. 54, No. 10, 913-925.
- Broder, A. (2002). A taxonomy of web search. SIGIR Forum, Vol. 36, No. 2, 3-10.
- Buckley, C. & Voorhees, E.M. (2000). Evaluating evaluation measure stability. In Proceedings of the 23rd annual international ACM SIGIR conference on Research and development in information retrieval, Athens, Greece, ACM Press, New York, 33-40.
- Burns, E. (2007). U.S. Search Engine Rankings, April 2007. Search Engine Watch. http://searchenginewatch.com/showPage.html?page=3626021
- Chu, H. & Rosenthal, M. (1996). Search Engines for the World Wide Web: A Comparative Study and Evaluation Methodology. Proceedings of the 59th ASIS Annual Meeting. http://www.asis.org/annual-96/ElectronicProceedings/chu.html
- Crystal, A. & Greenberg, J. (2006). Relevance Criteria Identified by Health Information Users During Web Searches. Journal of the American Society for Information Science and Technology, Vol. 57, No. 10, 1368-1382.
- Della Mea, V. & Mizzaro, S. (2004). Measuring Retrieval Effectiveness: A New Proposal and a First Experimental Validation. Journal of the American Society for Information Science and Technology, Vol. 55, No. 6, 530-543.
- Ding, W. & Marchionini, G. (1996). A comparative study of web search service performance. Proceedings of the 59th ASIS Annual Meeting, 136-142.
- Dresel, R., Hörnig, D., Kaluza, H., Peter, A., Roßmann, N. & Sieber, W. (2001). Evaluation deutscher Web-Suchwerkzeuge. Nachrichten für Dokumentation, Vol. 52, No. 7, 381-392.
- Gordon, M. & Pathak, P. (1999). Finding information on the World Wide Web: the retrieval effectiveness of search engines. Information Processing & Management, Vol. 35, No. 2, 141-180.
- Granka, L.A., Joachims, T., & Gay, G. (2004). Eye-tracking analysis of user behavior in WWW search. Proceedings of the 27th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval, 478-479.
- Griesbaum, J. (2004). Evaluation of three German search engines: Altavista.de, Google.de and Lycos.de. Information Research, Vol. 9, No. 4. http://informationr.net/ir/9-4/paper189.html
- Griesbaum, J., Rittberger, M. & Bekavac, B. (2002). Deutsche Suchmaschinen im Vergleich: AltaVista.de, Fireball.de und Lycos.de. Procedings of the 8. Internationales Symposium für Informationswissenschaft, 201-223.
- Guggenheim, E. & Bar-Ilan, J. (2005). Tauglichkeit von Suchmaschinen für deutschsprachige Anfragen. Information Wissenschaft und Praxis, Vol. 56, No. 1, 35-40.
- Gulli, A. & Signorini, A. (2005). The indexable Web is more than 11.5 billion pages. Proceedings of the 14th International Conference on World Wide Web, Chiba, Japan, 902-903.
- Hawking, D. & Craswell, N. (2005). The Very Large Collection and Web Tracks. In TREC: Experiment and Evaluation in Information Retrieval (S. 199-231). Cambridge, Mass.: MIT Press.
- Hawking, D., Craswell, N., Bailey, P. & Griffiths, K. (2001). Measuring Search Engine Quality. Information Retrieval, Vol. 4, No. 1, 33-59.
- Jacsó, P. (2005). Options for presenting search results. Online Information Review, Vol. 29, No. 3, 311-319.
- Järvelin, K. & Kekäiläinen, J. (2002). Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems, Vol. 20, No. 4, 422-446.
- Lazarinis, F. (2007). Web retrieval systems and the Greek language: do they have an understanding? Journal of Information Science, in press.
- Leighton, H.V. & Srivastava, J. (1999). First 20 Precision among World Wide Web Search Services (Search Engines). Journal of the American Society for Information Science, Vol. 50, No. 10, 870-881.
- Lewandowski, D. (2006). Query types and search topics of German Web search engine users. Information Services & Use, Vol. 26, No. 4, 261-269.
- Lewandowski, D. (2007a). Mit welchen Kennzahlen lässt sich die Qualität von Suchmaschinen messen? In Die Macht der Suchmaschinen / The Power of Search Engines (S. 243-258). Köln: von Halem.
- Lewandowski, D. (2007b). Trefferpräsentation in Web-Suchmaschinen. Proceedings of Information in Wissenschaft, Bildung und Wirtschaft; 29. Online-Tagung der DGI 2007, DGI, Frankfurt am Main, in press.
- Lewandowski, D. & Höchstötter, N. (2007). Web Searching: A Quality Measurement Perspective. In Web Searching: Interdisciplinary Perspectives (in press). Dordrecht: Springer.
- Lewandowski, D., Wahlig, H. & Meyer-Bautor, G. (2006). The Freshness of Web search engine databases. Journal of Information Science, Vol. 32, No. 2, 133-150.
- Machill, M., Neuberger, C., Schweiger, W. & Wirth, W. (2003). Wegweiser im Netz: Qualität und Nutzung von Suchmaschinen. In Wegweiser im Netz (S. 17-490). Gütersloh: Bertelsmann Stiftung.
- Mizzaro, S. (1997). Relevance: The Whole History. Journal of the American Society for Information Science, Vol. 48, No. 9, 810-832.
- Rose, D.E. & Levinson, D. (2004). Understanding user goals in Web search. Proceedings of the thirteenth International World Wide Web Conference, ACM Press, New York, 13-19.
- Schmidt-Maenz, N. & Koch, M. (2006). A General Classification of (Search) Queries and Terms. Proceedings of the 3rd International Conference on Information Technologies: Next Generations, Las Vegas, Nevada, USA, 375-381.
- Spink, A., Wolfram, D., Jansen, M. & Saracevic, T. (2001). Searching the Web: The public and their queries. Journal of the American Society for Information Science and Technology, Vol. 53, No. 2, 226-234.
- Stock, M. & Stock, W.G. (2000). Internet-Suchwerkzeuge im Vergleich, Teil 1: Retrievaltest mit Known Item Searches. Password, Vol. 15, No. 11, 23-31.
- Sullivan, D. (2006). Hitwise Search Engine Ratings. Search Engine Watch. http://searchenginewatch.com/showPage.html?page=3099931
- Sullivan, D. (2007). Major Search Engines and Directories. Search Engine Watch. http://searchenginewatch.com/showPage.html?page=2156221
- Tague-Sucliffe, J. (1992). The pragmatics of information retrieval experimentation, revisited. Information Processing & Management, Vol. 28, No. 4, 467-490.
- Vaughan, L. (2004). New Measurements for Search Engine Evaluation Proposed and Tested. Information Processing & Management, Vol. 40, No. 4, 677-691.
- Vaughan, L. & Thelwall, M. (2004). Search Engine Coverage Bias: Evidence and Possible Causes. Information Processing & Management, Vol. 40, No. 4, 693-707.
- Véronis, J. (2006). A comparative study of six search engines. http://www.up.univ-mrs.fr/veronis/pdf/2006-comparative-study.pdf
- Webhits Web-Barometer (2006). http://www.webhits.de/deutsch/index.shtml?webstats.html
- Wolff, C. (2000). Vergleichende Evaluierung von Such- und Metasuchmaschinen. Proceedings of 7. Internationales Symposium für Informationswissenschaft, Darmstadt, Germany, Universitätsverlag, Konstanz, 31-48.
|
