Information retrieval performance measures for a current awareness report composition aid

Krichel, Thomas Information retrieval performance measures for a current awareness report composition aid., 2006 [Preprint]

[thumbnail of sendai.pdf]
Preview
 PDF
sendai.pdf
Download (174kB) | Preview

English abstract

This papers studies a special ``small'' information retrieval problem where user satisfaction only depends on the ordering of documents. We look for a retrieval performance measure applicable for this setting. We define some requirements for such a measure. We develop a theoretical ordering of all outcomes. We look at some standard and purpose-build measures and assess them against the requirements. We conclude that a linear combination of two such measures is adequate.

Item type: Preprint
Keywords: information retrieval, performance evaluation, current awareness
Subjects: I. Information treatment for information services > II. Filtering.
Depositing user: Thomas Krichel
Date deposited: 01 Nov 2006
Last modified: 02 Oct 2014 12:03
URI: http://hdl.handle.net/10760/7675

References

Barrueco Cruz, J. M., Krichel, T., Trinidad Christensen, J. C., 2003. Organizing current awareness in a large digital library, presented at the 2003 Conference on Users in the Electronic Information Environments, in Espoo, Finland and at the II Jornadas de Tratamiento y Recuperacion de la Informacion

, in Leganes, Spain, both on September 8, 2003, available at http://openlib.org/home/krichel/papers/espoo.pdf.

Brookes, B. C., 1968. The measure of information retrieval effectiveness pro-posed by Swets. Journal of Documentation 24, 41­54.Cooper, W. S.,1968. Expec

ted search length: A single measure of retrieval effectiveness based on the weak ordering action of retrieval systems. Journal of American Society of Information Science 19, 30­41.

Cooper, W. S., 1973. On selecting a measure of retrieval effectiveness. part i the "subjective philosophy" of evaluation. Journal of the American Society for Information Science 24 (87­100).

Ginsparg, P., Houle, P., Joachims, T., Sul, J.-H., 2004. Mapping subsets of scholarly information. Proceedings of the ational Academy of Sciences of the USA 101, 5236­5240, available at http://arxiv.org/abs/cs.IR/0312018.

Joachims, T., 1999. Making large-scale svm learning practical. In: Schalkopf,B., Burges, C., Smola, A. J. (Eds.), Advances in Kernel Methods. Support Vector Learning. MIT Press.

Krichel, T., Bakkalbasi, N., 2005. Developing a predictive model of editor selectivity in a current awareness serviceof a large digital library. Library and Information Science Research 27 (4), 440­452.

Losee, R. M., 1998. Text Retrieval and Filtering: Analytic Models of Performance. Kluwer, Boston.Shaw, W. M., 1986. On the foundation of evaluation. Journal of the American Society for Information Science 37 (5), 346­348.

Swets, J. A., 1963. Information retrieval system. Science 141 (3577), 245­250.

Van Rijsbergen, C. K., 1974. Foundation of evaluation. Journal of Documentation 30 (4), 365­373.

Vapnik, V. N., 1995. The Nature of Statistical Learning Theory. Springer.


Downloads

Downloads per month over past year

Actions (login required)

View Item View Item
Contact us
Send email Sponsors
Sponsor: AIMS Sponsor: D&P Sponsor: UniNa CAB Get connected
Facebook Facebook Twitter Twitter Linkedin LinkedIn RSS RSS