Login
Create Account

The Influence of Commercial Intent of Search Results on Their Perceived Relevance

Lewandowski, Dirk The Influence of Commercial Intent of Search Results on Their Perceived Relevance., 2011 [Preprint]

Preview

PDF
iConference_preprint_2011.pdf
Download (468kB) | Preview

English abstract

We carried out a retrieval effectiveness test on the three major web search engines (i.e., Google, Microsoft and Yahoo). In addition to relevance judgments, we classified the results according to their commercial intent and whether or not they carried any advertising. We found that all search engines provide a large number of results with a commercial intent. Google provides significantly more commercial results than the other search engines do. However, the commercial intent of a result did not influence jurors in their relevance judgments.

Item type:	Preprint
Keywords:	Worldwide Web, search engines, commerciality, evaluation
Subjects:	L. Information technology and library technology > LC. Internet, including WWW. L. Information technology and library technology > LS. Search engines.
Depositing user:	Dirk Lewandowski
Date deposited:	27 Jun 2012
Last modified:	02 Oct 2014 12:22
URI:	http://hdl.handle.net/10760/17232

Check full metadata for this record

References

Borlund, P. (2003). The concept of relevance in IR. Journal of the American Society for Information Science and Technology, 54(10), 913-925.

Broder, A. (2002). A taxonomy of web search. SIGIR Forum, 36(2), 3-10.

Buckley, C., & Voorhees, E. M. (2000). Evaluating evaluation measure stability. SIGIR Forum (ACM Special Interest Group on Information Retrieval), 33-40.

Chu, H., & Rosenthal, M. (1996). Search engines for the World Wide Web: A comparative study and evaluation methodology. Proceedings of the Ninth Annual Meeting of the American Society for Information Science, Baltimore, Maryland, 21-4 Oct 1996. Edited by Steve Hardin. Medford, New Jersey: Information Today, Inc., p.127-35.

Couvering, E.V. (2007). Is relevance relevant? Market, science, and war: Discourses of search engine quality. Journal of Computer-Mediated Communication, 12(3), 866-887.

Cutrell, E., & Guan, Z. (2007). Eye tracking in MSN Search: Investigating snippet length, target position and task types. Microsoft Technical Report, MSR-TR-2007-01. ftp://ftp.research.microsoft.com/pub/tr/TR-2007-01.pdf

Ding, W., & Marchionini, G. (1996). A comparative study of web search service performance. Proceedings of the Ninth Annual Meeting of the American Society for Information Science, Baltimore, Maryland, 21-4 Oct 1996. Edited by Steve Hardin. Medford, New Jersey: Information Today, Inc., p.136-42.

Dresel, R., Hörnig, D., Kaluza, H., Peter, A., Roßmann, N., & Sieber, W. (2001). Evaluation deutscher Web-Suchwerkzeuge. Nachrichten für Dokumentation, 52(7), 381-392.

Gordon, M., & Pathak, P. (1999). Finding information on the World Wide Web: The retrieval effectiveness of search engines. Information Processing & Management, 35(2), 141-180.

Granka, L., Hembrooke, H., & Gay, G. (2005). Location, Location, Location: Viewing Patterns on WWW Pages. Proc. of Eye Tracking Research & Applications (ETRA): Symposium 2006, ACM

Griesbaum, J. (2004). Evaluation of three German search engines: Altavista.de, Google.de and Lycos.de. Information Research, 9(4). http://informationr.net/ir/9-4/paper189.html

Griesbaum, J., Rittberger, M., & Bekavac, B. (2002). In R. Hammwöhner, C. Wolff & C. Womser-Hacker (Eds.), Deutsche Suchmaschinen im Vergleich: AltaVista.de, Fireball.de, Google.de und Lycos.de (pp. 201-223). Paper presented at the Information und Mobilität. Optimierung und Vermeidung von Mobilität durch Information. 8. Internationales Symposium für Informationswissenschaft. UVK.

Hawking, D., & Craswell, N. (2005). The very large collection and Web tracks. In E. M. Voorhees & D.K. Harman (Eds.), TREC experiment and evaluation in information retrieval (pp. 199-231). Cambridge, Massachusetts: MIT Press.

Hawking, D., Craswell, N., Bailey, P., & Griffiths, K. (2001). Measuring search engine quality. Information Retrieval, 4(1), 33-59.

Hjorland, B. (2010). The foundation of the concept of relevance. Journal of the American Society for Information Science and Technology, 61(2), 217-237.

Hotchkiss, G. (2007). Search in the year 2010. Search Engine Land. http://searchengineland.com/search-in-the-year-2010-1191

Höchstötter, N., & Koch, M. (2008). Standard parameters for searching behaviour in search engines and their empirical evaluation. Journal of Information Science, 35(1), 45-64.

Höchstötter, N., & Lewandowski, D. (2009). What users see – Structures in search engine results pages. Information Sciences, 179(12), 1796-1812.

Jansen, B. J. (2007). The comparative effectiveness of sponsored and nonsponsored links for web e-commerce queries. ACM Transactions on the Web, 1(1), 1-25.

Jansen, B. J., & Molina, P. R. (2006). The effectiveness of Web search engines for retrieving relevant ecommerce links. Information Processing & Management, 42(4), 1075-1098.

Jansen, B. J., & Resnick, M. (2006). An examination of searcher's perceptions of nonsponsored and sponsored links during ecommerce Web searching. Journal of the American Society for Information Science and Technology, 57(14), 1949-1961.

Joachims, T., Granka, L., Pan, B., Hembrooke, H., and Gay, G. (2005). Accurately interpreting clickthrough data as implicit feedback. In Proceedings of the 28th Annual

international ACM SIGIR Conference on Research and Development in information Retrieval (Salvador, Brazil, August 15 - 19, 2005). SIGIR '05. ACM, New York, NY, 154-161.

Lazarinis, F., Vilares, J., Tait, J., & Efthimiadis, E. (2009). Current research issues and trends in non-English Web searching. Information Retrieval, 12(3), 230-250.

Leighton, H. V., & Srivastava, J. (1999). First 20 precision among World Wide Web search services (search engines). Journal of the American Society for Information Science, 50(10), 870-881.

Lewandowski, D. (2006). Query types and search topics of German Web search engine users. Information Services & Use, 26(4), 261-269.

Lewandowski, D. (2008). The retrieval effectiveness of Web search engines: Considering results descriptions. Journal of Documentation, 64(6), 915-937.

Lewandowski, D. (2009). The retrieval effectiveness of search engines on navigational queries. ASLIB Proceedings (in press).

Saracevic, T. (2007a). Relevance: A review of the literature and a framework for thinking on the notion in Information Science. Part II: Nature and manifestations of relevance. Journal of the American Society for Information Science and Technology, 58(13), 1915-1933.

Saracevic, T. (2007b). Relevance: A review of the literature and a framework for thinking on the notion in Information Science. Part III: Behavior and effects of relevance. Journal of the American Society for Information Science and Technology, 58(13), 2126-2144.

Sullivan, D. (2007). Major Search Engines and Directories. Retrieved 10.08.2010, from http://searchenginewatch.com/showPage.html?page=2156221

Tague-Sucliffe, J. (1992). The pragmatics of information retrieval experimentation, revisited. Information Processing & Management, 28(4), 467-490.

Vaughan, L., & Thelwall, M. (2004). Search Engine coverage bias: Evidence and possible causes. Information Processing & Management, 40(4), 693-707.

Vaughan, L., & Zhang, Y. (2007). Equal representation by search engines? A comparison of websites across countries and domains. Journal of Computer-Mediated Communication, 12(3), article 7.

Véronis, J. (2006). A comparative study of six search engines. Retrieved 10.08.2010, from http://www.up.univmrs.fr/veronis/pdf/2006-comparative-study.pdf

Webhits. (2010). Webhits Web-Barometer. Retrieved 10.08.2010, from http://www.webhits.de/deutsch/index.shtml?webstats.htm