REINA at CLEF 2007 Robust Track (2007)

Zazo, Ángel F. and G.-Figuerola, Carlos and Alonso-Berrocal, José-Luis REINA at CLEF 2007 Robust Track (2007)., 2007 . In CLEF 2007 Workshop, Budapest, Hungary, 19-21 September. [Conference paper]

Preview

PDF zazo2007reina.pdf Download (109kB) | Preview

English abstract

This paper describes our work at CLEF 2007 Robust Task. We have participated in the monolingual (English, French and Portuguese) and the bilingual (English to French) subtask. At CLEF 2006 our research group obtained very good results applying local query expansion using windows of terms in the robust task. This year we have used the same expansion technique, but taking into account some criteria of robustness: MAP, GMAP, MMR, GS@10, P@10, number of failed topics, number of topics bellow 0.1 MAP, and number of topics with P@10=0. In bilingual retrieval experiments three machine translation programs were used to translate topics. For the target language, translations were merged before performing a monolingual retrieval. We also applied the same local expansion technique. This year the results were disappointing. We think out that the reason is the difficulty to select the best measurement for robustness. Perhaps the problem is that all measurements are average results over all topics, but the hard topics are inherently hard and must be analyze separately. This year all our runs also ends up in good ranking, both base runs and expanded ones. We think that the reason is that we used a good information retrieval system, and the expansion technique is robust because it does not deteriorate significantly the retrieval performance.

Item type:	Conference paper
Keywords:	Robust Retrieval, Query Expansion, Term Windows, Association Thesauri, CLIR, Machine Translation, Traducción automática, Expansión de la búsqueda
Subjects:	L. Information technology and library technology > LM. Automatic text retrieval. L. Information technology and library technology > LL. Automated language processing.
Depositing user:	R. Gómez-Díaz
Date deposited:	17 Nov 2009
Last modified:	02 Oct 2014 12:15
URI:	http://hdl.handle.net/10760/13658

References

[1] G. M. Di Nunzio, N. Ferro, T. Mandl, and C. Peters. CLEF 2006: Ad hoc track overview.CLEF 2006, LNCS, 4730, 2007

[2] S. Tomlinson. Comparing the robustness of expansion techniques and retrieval measures. In A. Nardi, C. Peters, and J. Vicedo, editors, ABSTRACTS CLEF 2006 Workshop, 20-22 September, Alicante, Spain. Results of the CLEF 2006 Cross-Language System Evaluation Campaign, 2006

[3] E. M. Voorhees. Overview of the TREC 2003 robust retrieval track. In The Twelfth Text REtrieval Conference (TREC 2003), pages 69–77. NIST Special Publication 500-255, 2003

[4] E. M. Voorhees. Overview of the TREC 2004 robust retrieval track. In The Thirteen Text REtrieval Conference (TREC 2004), Gaithersburg, Maryland, November 16-19. NIST Special Publication 500-261, 2004

[5] E. M. Voorhees. Overview of the TREC 2005 robust retrieval track. In The Fourteenth TextREtrieval Conference (TREC 2005), Gaithersburg, Maryland, November 15-18. NIST, 2005.

[6] A. F. Zazo, J. L. Alonso Berrocal, and C. G. Figuerola. Local query expansion using terms windows for robust retrieval. CLEF 2006, LNCS, 4730:145–152, 2007.

Downloads

Actions (login required)

View Item