E-LIS, Eprints in Library and Information Science Homepage E-LIS, Eprints in Library and Information Science
   home   |   about   |   search   |   browse   |   register   |   registered users area   |   help   |   FAQ   |   JITA   

Information Retrieval Effectiveness of Turkish Search Engines

Bitirim, Yıltan and Tonta, Yaşar and Sever, Hayri (2002) Information Retrieval Effectiveness of Turkish Search Engines. In Yakhno, Tatyana, Eds. Proceedings Advances in Information Systems: Second International Conference, ADVIS 2002, pp. 93-103, İzmir (Turkey).

Full text available as:
PDF - Requires Adobe Acrobat Reader or other PDF viewer.

Abstract

This is an investigation of information retrieval performance of Turkish search engines with respect to precision, normalized recall, coverage and novelty ratios. We defined seventeen query topics for Arabul, Arama, Netbul and Superonline. These queries were carefully selected to assess the capability of a search engine for handling broad or narrow topic subjects, exclusion of particular information, identifying and indexing Turkish characters, retrieval of hub/authoritative pages, stemming of Turkish words, correct interpretation of Boolean operators. We classified each document in a retrieval output as being ”relevant” or ”nonrelevant” to calculate precision and normalized recall ratios at various cut-off points for each pair of query topic and search engine. We found the coverage and novelty ratios for each search engine.We also tested how search engines handle meta-tags and dead links. Arama appears to be the best Turkish search engine in terms of average precision and normalized recall ratios, and the coverage of Turkish sites. Turkish characters (and stemming as well) still cause bottlenecks for Turkish search engines. Superonline and Netbul make use of the indexing information in metatag fields to improve retrieval results.

Keywords:Information retrieval performance, Turkish search engines, precision, recall, coverage, novelty
Subjects:L. Information technology and library technology. > LS. Search engines.
ID Code:9687
Deposited By:tonta, yaşar
Deposited On:09 May 2007
Alternative Locations:http://yunus.hacettepe.edu.tr/~tonta/yayinlar/se.pdf
All fields:Show all fields

1. M. Kobayashi and K. Takeda. Information retrieval on the web. ACM Computing Surveys, 32(2):144–172, June 2000.

2. J. Jansen. Using an intelligent agent to enhance search engine performance. First Monday, 1996. http://www.firstmonday.dk/issues/issue2 3/jansen/index.html.

3. W. Mettrop and P. Nieuwenhuysen. Internet search engines: Fluctuations in document accessibility. Journal of Documentation, 57:623–651, 2001.

4. W.B. Croft and H. Turtle. A retrieval model for incorporating hypertext links. In Proceedings of ACM Hypertext Conference, pages 213–224, New Orleans, LA, November 1989.

5. V.N. Gudivada, V.V. Raghavan, W.I. Grosky, and R. Kasanagottu. Information retrieval on the world wide web. IEEE Internet Computing, 1(5):58–68, 1997.

6. H. Chu and M. Rosenthal. Search engines for the world wide web: A comparative study and evaluation methodology. In Steve Hardin, editor, Proceedings of the 59th ASIS Annual Meeting, pages 127–135, Baltimore, Maryland, October 1996.

7. H.V. Lerghton and J.V. Srivastava. First 20 precision amongWWWsearch services. Journal of the American Society for Information Science, 50:870–881, 1999.

8. C. Oppenheim, A. Morris, and C. McKnight. The evaluation of WWW search engines. Journal of Documentation, 56:190–211, 2000.

9. J. Savoy and J. Picard. Retrieval effectiveness on the web. Information Processing and Management, 37:543–569, 2001.

10. M. Gordon and P. Pathak. Finding information on the WWW: The retrieval effectiveness of search engines. Information Processing and Management, 35:141–180, 1999.

11. J. S. Deogun, H. Sever, and V. V. Raghavan. Structural abstractions of hypertext documents for web-based retrieval. In Roland R. Wagner, editor, Proceedings of Ninth International Workshop on Database and Expert Systems Applications, (in conjunction with DEXA’98), pages 385–390, Vienna, Austria, August 1998.

12. M.E. Küçük, B. Olgun, and H. Sever. Application of metadata concepts to discovery of internet resources. In Tatyana Yakhno, editor, Advances in Information Systems (ADVIS’00), volume 1909, pages 304–313. Springer Verlag, Berlin, GR, October 2000.

13. J.M. Kleinberg. Authoritative source in a hyperlinked environment. In Proceedings of the 9th Annual ACM-SIAM Symposium on Discrete Algorithms, pages 668–677, 1998.

14. S. C. Clarke and P. Willet. Estimating the recall performance of web search engines. Aslib Proceedings, 49(7):184–189, July/August 1997.

15. D. Hawking, N. Craswell, P. Thislewaite, and D. Harman. Results and challenges in web search evaluation. In D. Harman, editor, Proceedings of the 8th Text REtrieval Conference (TREC-8), Gaithersburg, Maryland, November 1999.

16. Y.Y. Yao. Measuring retrieval effectiveness based on user preference of documents. Journal of the American Society for Information Science, 46:133–145, 1995.

17. W.B. Cooper. Expected search length: A single measure of retrieval effectiveness based on the weak ordering action of retrieval systems. American Documentation, 19:30–41, 1968.

18. S. Lawrence and C.L. Giles. Searching the world wide web. Science, 280(5360):98–100, 3 April 1998. http://www.neci.nec.com/ lawrence/science98.html.

19. R.R. Korfhage. Information Storage and Retireval. Wiley, New York, NY, 1997.

20. B. Jansen, A. Spink, J. Bateman, and T. Saracevic. Real life information retrieval: A study of user queries on the web. SIGIR Forum, 32(1):5–17, 1998.

21. C. Silverstein, M. Henziger, H. Marais, and M. Moricz. Analysis of a very large web search engine query log. SIGIR Forum, 33(1):6–12, 1999.

Archive Staff Only: edit this record

   home   |   about   |   search   |   browse   |   register   |   registered users area   |   help   |   FAQ   |   JITA   

Send comments and questions to      |     E-LIS is hosted by AEPIC at CILEA      |     Powered by GNU Eprints software